<---- template headericclude ----->
Need actual text LOG files - Journald & Syslog-ng not talking to each other
FedoraForum.org - Fedora Support Forums and Community
Results 1 to 6 of 6
  1. #1
    Join Date
    Mar 2017
    Location
    USA - Virginia
    Posts
    2
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Unhappy [SOLVED] Need actual text LOG files - Journald & Syslog-ng not talking to each other

    Journalctl / journald logs aren't going to work for me because I need individual log files that can sent to different areas of our infrastructure for processing / archiving and a ton of other activities. Basically it is a non-starter to change the way the workflow around these logs work.

    I have syslog-ng installed because the back-end systems us it to centralize files. I can't change that either - so syslog-ng is what I have to use.

    It is professed that syslog-ng can read journald directly with the system() source, but its just not happening. I see the logging information in journalctl, but can't get it to simply WRITE to a disk location to save my life. Most of the work going on is Perl, executed in cron - this works fine on other machines not running systemd/journald. It can't be this hard - what am I missing?

    Thanks in advance - Please see my configs below

    (journald.conf is default - no lines uncommented.. and Fedora 25's systemd doesn't seem to honor the ForwardToSyslog switch anyway.)

    syslog-ng.conf
    Code:
    @version:3.9
    @include "scl.conf"
    
    # syslog-ng configuration file.
    #
    # This should behave pretty much like the original syslog on RedHat. But
    # it could be configured a lot smarter.
    #
    # See syslog-ng(8) and syslog-ng.conf(5) for more information.
    #
    # Note: it also sources additional configuration files (*.conf)
    #       located in /etc/syslog-ng/conf.d/
    
    options {
        flush_lines (0);
        time_reopen (10);
        log_fifo_size (1000);
        chain_hostnames (off);
        use_dns (no);
        use_fqdn (no);
        create_dirs (no);
        keep_hostname (yes);
    };
    
    source s_sys {
        system();
        internal();
        # udp(ip(0.0.0.0) port(514));
    };
    
    destination d_cons { file("/dev/console"); };
    destination d_mesg { file("/var/log/messages"); };
    destination d_auth { file("/var/log/secure"); };
    destination d_mail { file("/var/log/maillog" flush_lines(10)); };
    destination d_spol { file("/var/log/spooler"); };
    destination d_boot { file("/var/log/boot.log"); };
    destination d_cron { file("/var/log/cron"); };
    destination d_kern { file("/var/log/kern"); };
    destination d_mlal { usertty("*"); };
    
    filter f_kernel     { facility(kern); };
    filter f_default    { level(info..emerg) and
                            not (facility(mail)
                            or facility(authpriv) 
                            or facility(cron)); };
    filter f_auth       { facility(authpriv); };
    filter f_mail       { facility(mail); };
    filter f_emergency  { level(emerg); };
    filter f_news       { facility(uucp) or
                            (facility(news) 
                            and level(crit..emerg)); };
    filter f_boot   { facility(local7); };
    filter f_cron   { facility(cron); };
    filter f_appservers { facility(local4); host("(^jenkins|app$|app_d$)"); };
    
    # Custom additions to support application
    destination d_speciallogs { file("/var/log/perl_logs/${PROGRAM}.log" create_dirs(yes) owner("automagic") group("automagic") perm(0660) dir-owner("automagic") dir-group("automagic") dir-perm(770) ); };
    destination d_DEBUG { file("/var/log/perl_logs/DEBUG.log" create_dirs(yes) owner("automagic") group("automagic") perm(0660) dir-owner("automagic") dir-group("automagic") dir-perm(770) ); };
    destination d_INFO { file("/var/log/perl_logs/SYSINFO.log" create_dirs(yes) owner("automagic") group("automagic") perm(0660) dir-owner("automagic") dir-group("automagic") dir-perm(770) ); };
    destination d_ERROR { file("/var/log/perl_logs/ERROR.log" create_dirs(yes) owner("automagic") group("automagic") perm(0660) dir-owner("automagic") dir-group("automagic") dir-perm(770) ); };
    destination d_central_syslog { syslog("logcentral.server" transport("tcp")); };
    
    #log { source(s_sys); filter(f_kernel); destination(d_cons); };
    log { source(s_sys); filter(f_kernel); destination(d_kern); };
    log { source(s_sys); filter(f_default); destination(d_mesg); };
    log { source(s_sys); filter(f_auth); destination(d_auth); };
    log { source(s_sys); filter(f_mail); destination(d_mail); };
    log { source(s_sys); filter(f_emergency); destination(d_mlal); };
    log { source(s_sys); filter(f_news); destination(d_spol); };
    log { source(s_sys); filter(f_boot); destination(d_boot); };
    log { source(s_sys); filter(f_cron); destination(d_cron); };
    
    
    # Modified for custom Application
    log { source(s_sys); filter(f_appservers); destination(d_central_syslog); destination(d_speciallogs); };
    log { source(s_sys); filter(f_appservers); filter( f_DEBUG ); destination(d_DEBUG); };
    log { source(s_sys); filter(f_appservers); filter( f_INFO ); destination(d_INFO); };
    log { source(s_sys); filter(f_appservers); filter( f_ERROR ); destination(d_ERROR); };
    
    filter f_DEBUG { level(debug..emerg); };
    filter f_INFO { level(info..emerg); };
    filter f_ERROR { level(err..emerg); };
    
    # Source additional configuration files (.conf extension only)
    @include "/etc/syslog-ng/conf.d/*.conf"
    
    
    # vim:ft=syslog-ng:ai:si:ts=4:sw=4:et:
    Last edited by cuttenweiler; 22nd March 2017 at 05:52 PM. Reason: Marked SOLVED

  2. #2
    PabloTwo's Avatar
    PabloTwo is offline "Fedora User" T-Shirt Winner
    Join Date
    Mar 2007
    Location
    Seville, FL
    Posts
    8,903
    Mentioned
    39 Post(s)
    Tagged
    0 Thread(s)

    Re: Need actual text LOG files - Journald & Syslog-ng not talking to each other

    You can have the traditional plain text /var/log/messages* files by installing the rsyslog package.

  3. #3
    Join Date
    Mar 2017
    Location
    Hungary
    Posts
    2
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Need actual text LOG files - Journald & Syslog-ng not talking to each other

    Hi,

    Try the following, and see if it helps:
    * Try temporarily disable Selinux/apparmor. If it helps, adjust your selinux/apparmor settings to allow syslog-ng to write the log files.
    * Try to remove the host filter from the f_appservers filter. I'm not sure that the host information is available in the journal.
    * The f_DEBUG and other filters are defined after they are used. It might not be an issue, but move them above the log statements.

    HTH,

    Robert

  4. #4
    PabloTwo's Avatar
    PabloTwo is offline "Fedora User" T-Shirt Winner
    Join Date
    Mar 2007
    Location
    Seville, FL
    Posts
    8,903
    Mentioned
    39 Post(s)
    Tagged
    0 Thread(s)

    Re: Need actual text LOG files - Journald & Syslog-ng not talking to each other

    I guess I didn't pay enough attention to this statement:
    (journald.conf is default - no lines uncommented.. ...
    The default is:
    Code:
    grep ToSyslog /etc/systemd/journald.conf 
    #ForwardToSyslog=no

  5. #5
    Join Date
    Mar 2017
    Location
    USA - Virginia
    Posts
    2
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Need actual text LOG files - Journald & Syslog-ng not talking to each other

    Robert for the win!

    The issue was that the journald does not handle the host entry in the f_appservers filter!

    So I changed this:

    Code:
    filter f_appservers { facility(local4); host("(^jekins|app$|app_d$)"); };
    To this:

    Code:
    filter f_appservers { facility(local4); };
    ... and it all started working.

    Right before you replied I noticed that everything was going to the /var/log/messages, just not the specified destinations, so I was going to " syslog-ng -Fevd" to trace the debug once I noticed that, but your suggestion helped keep me out of the weeds.

    Also, apparently syslog-ng.conf is evaluated as a whole, not in sequenced order. I inherited this file, but I'll be re-arranging it to be more human readable as well.

    I really can't thank you enough Robert!

    Cheers!


    Quote Originally Posted by frobert
    Hi,

    Try the following, and see if it helps:
    * Try temporarily disable Selinux/apparmor. If it helps, adjust your selinux/apparmor settings to allow syslog-ng to write the log files.
    * Try to remove the host filter from the f_appservers filter. I'm not sure that the host information is available in the journal.
    * The f_DEBUG and other filters are defined after they are used. It might not be an issue, but move them above the log statements.

    HTH,

    Robert

  6. #6
    Join Date
    Mar 2017
    Location
    Hungary
    Posts
    2
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Re: Need actual text LOG files - Journald & Syslog-ng not talking to each other

    I'm really glad that it's working now, let us know if you need further help with syslog-ng!

Similar Threads

  1. Replies: 2
    Last Post: 19th February 2016, 06:02 PM
  2. F19 Logrotate of syslog files not working
    By cmaxeiner in forum Using Fedora
    Replies: 3
    Last Post: 13th May 2014, 09:27 AM
  3. journald and syslog question?
    By fsckPotthead in forum Using Fedora
    Replies: 1
    Last Post: 9th May 2014, 03:45 AM
  4. SVG Files - no text.
    By Adunaic in forum Using Fedora
    Replies: 2
    Last Post: 1st November 2010, 04:18 PM
  5. finding files with wildcards in the actual filename
    By kyle775 in forum Using Fedora
    Replies: 2
    Last Post: 5th September 2010, 05:26 AM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
[[template footer(Guest)]]