Scribe

https://github.com/AMTM-OSR/scribe

https://github.com/AMTM-OSR/uiScribe

RFC3164 Network/Compute Devices -> syslog-ng (UDP port 514) -> Promtail (port 1514) -> Loki (port 3100) <- Grafana (port 3000)

Syslog-ng — принимать логи по сети от других устройств и систем.

Loki — обрабатывать логи и отправлять их в Grafana для визуализации.

Grafana — визуализировать логи, собранные с помощью Loki.

Важно, что Syslog-ng, Loki и Grafana могут быть на одном сервере для оптимальной производительности, но Grafana может быть на отдельном сервере.

Настроить приём логов от устройств, например, по UDP-порту 514 или TCP-порту. В конфигурации syslog-ng можно указать источник логов и вывод в файл. Например, для приёма логов от MikroTik syslog-ng может слушать UDP-порт 5140 и записывать логи в JSON-файл. Настроить отправку логов на удалённый сервер, если устройства отправляют syslog с отклонением от формата RFC 3164 (например, устройства Cisco). В этом случае syslog-ng может преобразовывать syslog к формату RFC 5424.

Подготовка

$ amtm ep /tmp/mnt/SYS

Установка

$ /usr/sbin/curl --retry 3 "https://raw.githubusercontent.com/AMTM-OSR/scribe/master/scribe.sh" -o "/jffs/scripts/scribe" && chmod 0755 /jffs/scripts/scribe && /jffs/scripts/scribe install

Настройка

nano /opt/etc/syslog-ng.conf

############################################################################# # syslog-ng.conf customized for scribe on Asuswrt-Merlin firmware # compare to /opt/share/syslog-ng/examples/syslog-ng.conf-opkg for differences from Entware distribution # # syslog-ng documentation: https://www.syslog-ng.com/technical-documents/list/syslog-ng-open-source-edition # # Release notes: https://github.com/syslog-ng/syslog-ng/releases @version: 4.7 #@include "scl.conf" # uncomment this line to for additional functionality, see syslog-ng documentation @include "/opt/etc/syslog-ng.d/" # Put any customization files in this directory options { chain_hostnames(no); # Enable or disable the chained hostname format. create_dirs(yes); keep_hostname(yes); # Enable or disable hostname rewriting. log_fifo_size(256); # The number of messages that the output queue can store. log_msg_size(16384); # Maximum length of a message in bytes. stats(freq(21600)); # The period between two STATS messages sent by syslog-ng, containing statistics about dropped logs in seconds; 0 disables. (21,600 seconds = 6 hours) flush_lines(0); # How many lines are flushed to a destination at a time. use_fqdn(no); # Add Fully Qualified Domain Name instead of short hostname. }; # syslog-ng gets messages from the system, kernel, and syslog-ng (internal) # DO NOT use system() source; causes issues on HND routers # so_rcvbuf = maximum number of messages per second * 1024 source src { unix-dgram("/dev/log" so_rcvbuf(65536) flags(syslog-protocol)); file("/proc/kmsg" program_override("kernel") flags(kernel)); internal(); # udp(ip(192.168.x.y) port(514)); # uncomment this line to pass all network messages through syslog-ng filters }; # if you only want to pass network messages through some syslog-ng filters, uncomment the source line below # then add "source(net);" to the log statement in any filter you want to pass network messages through #source net { udp(ip(192.168.x.y) port(514)); }; # set the filename for the default log file - anything not filtered out will end up here destination messages { file("/opt/var/log/messages"); }; # to send log messages to the local network, uncomment the destination line below # then add "destination(log_server);" to the log statement in any filter you want to pass network messages through #destination log_server { udp("192.168.x.y" port(514)); }; log { source(src); # source(net); # uncomment this and "source net" function above to get udp log messages from local network destination(messages); # destination(log_server); # uncomment this and "destination log_server" function above to send udp log messages to local network };

меняем функцию log на

destination d_fluentbit { syslog("192.168.1.132" transport("tcp") port(5140)); }; log { source(src); # source(net); # uncomment this and "source net" function above to get udp log messages from local network destination(messages); # destination(log_server); # uncomment this and "destination log_server" function above to send udp log messages to local network destination(d_fluentbit); };