Scala 语言实战：日志分析工具——统计错误日志 TOP N 异常类型
在软件开发过程中，日志是记录程序运行状态的重要信息来源。通过对日志的分析，我们可以快速定位问题、优化性能、提高系统的稳定性。本文将使用 Scala 语言，结合实际案例，实现一个日志分析工具，用于统计错误日志中 TOP N 异常类型。
环境准备
在开始编写代码之前，我们需要准备以下环境：
1. Scala 语言环境：下载并安装 Scala，配置环境变量。
2. sbt 构建工具：用于构建 Scala 项目。
3. IntelliJ IDEA 或其他 Scala 集成开发环境。
项目结构
我们的项目结构如下：
log-analysis
├── src
│ ├── main
│ │ ├── scala
│ │ │ └── LogAnalysisApp.scala
│ └── test
│ ├── scala
│ │ └── LogAnalysisAppSpec.scala
├── build.sbt
└── README.md
1. 定义日志格式
我们需要定义日志的格式。以下是一个常见的错误日志格式：
[INFO] 2023-03-01 10:00:00 - User: admin, Operation: login, Result: success
[ERROR] 2023-03-01 10:05:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password
2. 解析日志
接下来，我们需要编写一个函数来解析日志，并提取出错误信息。以下是一个简单的解析函数：
scala
object LogParser {
def parseErrorLog(log: String): Option[(String, String)] = {
val pattern = """[(ERROR)] (.?) - (.?), Result: (.?), Error: (.?)$""".r
pattern.findFirstMatchIn(log).map { m =>
(m.group(2), m.group(5))
}
}
}
3. 统计异常类型
现在我们已经解析出了错误信息，接下来我们需要统计这些错误信息中 TOP N 的异常类型。以下是一个统计函数：
scala
object ErrorStats {
def topNErrorTypes(logs: Seq[String], n: Int): Map[String, Int] = {
val errorTypes = logs.flatMap(LogParser.parseErrorLog)
val errorCounts = errorTypes.groupBy(_._2).mapValues(_.size)
errorCounts.toList.sortBy(-_._2).take(n).toMap
}
}
4. 主程序
我们需要编写主程序来读取日志文件，并输出 TOP N 的异常类型。以下是一个简单的示例：
```scala
object LogAnalysisApp extends App {
val logs = Seq(
"[INFO] 2023-03-01 10:00:00 - User: admin, Operation: login, Result: success",
"[ERROR] 2023-03-01 10:05:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 10:10:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 10:15:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 10:20:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 10:25:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 10:30:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 10:35:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 10:40:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 10:45:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 10:50:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 10:55:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 11:00:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 11:05:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 11:10:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 11:15:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 11:20:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 11:25:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 11:30:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 11:35:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 11:40:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 11:45:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 11:50:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 11:55:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 12:00:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 12:05:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 12:10:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 12:15:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 12:20:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 12:25:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 12:30:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 12:35:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 12:40:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 12:45:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 12:50:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 12:55:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 13:00:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 13:05:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 13:10:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 13:15:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 13:20:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 13:25:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 13:30:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 13:35:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 13:40:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 13:45:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 13:50:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 13:55:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 14:00:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 14:05:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 14:10:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 14:15:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 14:20:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 14:25:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 14:30:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 14:35:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 14:40:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 14:45:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 14:50:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 14:55:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 15:00:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 15:05:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 15:10:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 15:15:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 15:20:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 15:25:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 15:30:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 15:35:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 15:40:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 15:45:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 15:50:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 15:55:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 16:00:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 16:05:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 16:10:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 16:15:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 16:20:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 16:25:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 16:30:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 16:35:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 16:40:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 16:45:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 16:50:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 16:55:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 17:00:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 17:05:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 17:10:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 17:15:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 17:20:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 17:25:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 17:30:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 17:35:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 17:40:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 17:45:00 - User: guest, Operation: login, Result: failed, Error: Invalid username or password",
"[ERROR] 2023-03-01 17:50:00 - User: guest, Operation: login
Comments NOTHING