Get data from Login Audit.csv in distributed environment?

lHi, 

Is there any way to achieve below scenario other than log streaming form Appain?

Currently our PROD environment has 3 servers and based on the load users are navigated to different Servers. 

We would like to get login-audit.csv file from all the 3 servers. Currently with readcsvlog function we are able to get data only from the current server where the function is ran.

Any suggestions are appreciated. 

Thanks.

  Discussion posts and replies are publicly visible

Parents
  • 0
    Certified Lead Developer

    I have a process that does uses the "read log" plug-in function.  I found in our distribued prod environment that it had approximately a random chance of pulling the logs from one of the 3 distributed log directories if I ran several in a row.

    So I made a separate process, with a "start process" smart service node calling this subprocess (and passing its data back to the parent).  I then set this node to run on MNI 10 times, "run one at a time" (important).  All the data gets passed back to the parent process and then deduplicated - and I have proven that this will pull from all 3 login-audit.csv files, when run a sufficient number of times.  Even then, this information should be considered "approximate".

    Also keep in mind that if the environment has fairly few logins for the current day, the login-audit.csv file in some of the distributed directories might still be the data for a previous day. So whatever you implement will need to account for this possibility as well.

  • Thanks, Will try it out. Can you suggest how we need to pull the previous day log? 
    Currently I'm using below code and it is working fine. 

    queryappianlogs(
          sqlStatement: "SELECT * FROM login-audit",
          hasHeader: false()
        ).data
    But when I tried to use this it didn't return any value 
    queryappianlogs(
          sqlStatement: "SELECT * FROM login-audit.csv.2020-03-16",
          hasHeader: false()
        ).data

  • 0
    Certified Lead Developer
    in reply to adithyay

    I'm using the "Log Reader" plug-in which handles it slightly differently and, slightly better IMHO.  With that instead of writing SQL statements you can just pull the contents of a named log file.

    I'm not entirely sure what you mean by "suggest how we need to pull" - can you clarify?

  • 0
    Certified Lead Developer
    in reply to adithyay

    In general you just include the date in the filename like you've done above.  But for an environment with distributed logs, your engine 1 might have a file matching yesterday's date, while engine 2 still has yesterday's logins stored in the general "login-audit.csv".

    So I set up something in my subprocess with one extra level of complexity - I have it query the named log file for yesterday, then i check if the results were blank, and if so, i query the general login audit file.

    In my case, after deduplication and all, I write these to my own database table.  But because of the possibility for duplicated entries to be written, I then take the further step of querying the existing table for matching entries and weeding out ones that have already been written (i.e. entries with the same username and login timestamp).

  • We are using the Log-Reader plug-in too. Our cron job just copies over the log-in.csv from past day to the Appian//log-folder.
    Appian renames the log-in files every day, as explained here: https://docs.appian.com/suite/help/19.4/Logging.html#managing-log-files

  • 0
    Certified Lead Developer
    in reply to juergeng393

    If you're using an on-prem install where you can just copy the log file, though, I don't believe you will have the same issues with multiple distributed servers as mentioned in the original post.  If you do and you've found a way around this, I'd be curious to hear what technique you used.

Reply Children