基于效率导向的自适应大语言模型日志解析方法

EFFICIENCY GUIDED ADAPTIVE LOG PARSING USING LARGE LANGUAGE MODEL

  • 摘要: 日志数据包含了软件服务运行过程中关键的行为信息,因而具有重要的研究与应用价值。日志解析作为日志处理流程中的核心步骤,通过将半结构化数据转化为结构化数据,显著提升了对日志信息的分析能力和利用效率。然而,现有基于大语言模型的日志解析方法存在冷启动困难和效率低下的问题。基于效率导向的自适应解析方法(EGAP)通过在传统解析方法的基础上,引入大语言模型的在线优化策略,有效提升了解析的精度与效率。EGAP利用模板缓存机制实现日志模板的快速匹配,并通过效率估算机制,灵活控制大语言模型的使用,以确保解析过程在高效性和准确性之间取得平衡。实验结果表明,EGAP在显著提升日志解析准确性的同时,大幅提高了解析效率。

     

    Abstract: Log data contains critical information about the runtime behavior of software services, making it highly valuable for research and application. Log parsing, as a core step in the log processing workflow, converts semi-structured data into structured data, significantly enhancing the analysis and utilization of log information. However, existing log parsing methods based on large language models face challenges such as cold start difficulties and inefficiency. The efficiency-guided adaptive parsing (EGAP) method introduced an online optimization strategy using large language models, based on traditional parsing methods, to effectively improve parsing accuracy and efficiency. EGAP used a template caching mechanism for rapid log template matching and employed an efficiency estimation mechanism to dynamically control the use of large language models, ensuring a balance between efficiency and accuracy in the parsing process. Experimental results demonstrate that EGAP significantly enhances log parsing accuracy while substantially improving parsing efficiency.

     

/

返回文章
返回