VRL

Vector Remap Language

良いところ

エラーログが親切すぎて、問題を把握しやすいのが助かる

欲しいと思った機能はだいたい揃っているので、自分で頑張るより組み込み関数を探した方が良い

悪いところ

code:vrl

structured = parse_syslog!(.message)

. = merge(., structured)

お試し

code:shell

docker run -it --rm timberio/vector:latest-alpine

code:shell

$ docker run -it --rm timberio/vector:latest-alpine vrl

@shimizukawa ➜ /workspaces/bpredmine (master) $ docker run -it --rm timberio/vector:latest-alpine vrl

VVVVVVVV VVVVVVVVRRRRRRRRRRRRRRRRR LLLLLLLLLLL

V::::::V V::::::VR::::::::::::::::R L:::::::::L

V::::::V V::::::VR::::::RRRRRR:::::R L:::::::::L

V::::::V V::::::VRR:::::R R:::::RLL:::::::LL

V:::::V V:::::V R::::R R:::::R L:::::L

V:::::V V:::::V R::::R R:::::R L:::::L

V:::::V V:::::V R::::RRRRRR:::::R L:::::L

V:::::V V:::::V R:::::::::::::RR L:::::L

V:::::V V:::::V R::::RRRRRR:::::R L:::::L

V:::::V V:::::V R::::R R:::::R L:::::L

V:::::V:::::V R::::R R:::::R L:::::L

V:::::::::V R::::R R:::::R L:::::L LLLLLL

V:::::::V RR:::::R R:::::RLL:::::::LLLLLLLLL:::::L

V:::::V R::::::R R:::::RL::::::::::::::::::::::L

V:::V R::::::R R:::::RL::::::::::::::::::::::L

VVV RRRRRRRR RRRRRRRLLLLLLLLLLLLLLLLLLLLLLLL

VECTOR REMAP LANGUAGE

Welcome!

The CLI is running in REPL (Read-eval-print loop) mode.

To run the CLI in regular mode, add a program to your command.

VRL REPL commands:

help Learn more about VRL

next Load the next object or create a new one

prev Load the previous object

exit Terminate the program

Any other value is resolved to a VRL expression.

$ parsed = parse_nginx_log(log, "combined")

┌─ :1:10

│

1 │ parsed = parse_nginx_log(log, "combined")

│ -------- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

│ │ │

│ │ this expression is fallible

│ │ update the expression to be infallible

│ or change this to an infallible assignment:

│ parsed, err = parse_nginx_log(log, "combined")

│

$ parsed = parse_nginx_log!(log, "combined")

$ .http = parsed

$ .

$ flatten(.)

code:otel-collector.yml

filelog/nginx/access:

include:

- /var/log/nginx/access.log

start_at: end

attributes:

service: nginx

env: prod

operators:

- type: regex_parser

parse_to: attributes.http

timestamp:

parse_from: attributes.http.time

layout_type: strptime

layout: "%d/%b/%Y:%H:%M:%S %z"

severity:

parse_from: attributes.http.status_code

preset: none

mapping:

info:

- 2xx

- 3xx

warn: 4xx

error: 5xx

- type: remove

field: attributes.http.time

- type: uri_parser

parse_from: attributes.http.path

code:vector.toml

type = "file"

read_from = "end"

type = "remap"

source = '''

.http = parse_nginx_log(.message, "combined") ?? parse_nginx_log!(.message, "error")

path_parts = split(string(.http.path), "?", limit: 2)

status = to_int!(.http.status)

.severity = if (status<400){"info"} else if (status<500){"warn"} else {"error"}

.timestamp = .http.timestamp || now()

. = flatten(.)

."service" = "nginx"

'''

httpのstatus_codeをseverityにマッピングする関数くらいあるんじゃないの？と思うけど、今のところ見つからない