Multiple Unnecessary Error Logs from Appian Excel Tools plugin [ERROR com.appiancorp.ps.exceltools.util.DatabaseUtils - Database name is: ]

Certified Lead Developer

Hello All,

We are using Appian Excel Tools com.appiancorp.ps.exceltools (v1.9.6) [com.appiancorp.ps.exceltools] - Smart Service => Import CSV to Database v3 to import csv data to database tables.

The data is being loaded successfully but we have millions of records which are being loaded through this plugin. This is leading us to another issue which is caused due to a small bug in the plugin.

The plugin's DatabaseUtils class has a method 'getDatabaseType' which has a line of code which writes to error log. It looks like a debug log but has wrongfully designated as error log. We have suppressed debug logs but since this message is being written as error logs our log files are having millions of line of debug statements wrongfully categorized as Error.

See the error log below - we have millions of line of the below message in our log files (basically equal to number of records inserted using this plugin)

 

The code snippet from the plugin is as follows which is causing the problem:

public static DatabaseType getDatabaseType(String dbName) {

                                LOG.error("Database name is: " + dbName);

                                if (DatabaseType.ORACLE.matches(dbName)) {

                                                return DatabaseType.ORACLE;

                                } else if (DatabaseType.MSSQL.matches(dbName)) {

                                                return DatabaseType.MSSQL;

                                } else if (DatabaseType.DB2.matches(dbName)) {

                                                return DatabaseType.DB2;

                                } else {

                                                return DatabaseType.MYSQL;

                                }

                }

The above highlighted statement should be LOG.debug("Database name is: " + dbName);

Due to this large CSV files are failing aswell since the buffer is not able to write to the log files with increase in log file size to 1+ GB.

We cannot suppress Error logs so please suggest what can we do to resolve the issue. Can we suppress error logs for this particular plugin as quick fix until the fix for this issue is published in newer version.

Any help / suggestion is appriciated.

Thanks,

Ketan

  Discussion posts and replies are publicly visible

  • Hi Ketan,
    I believe you are using the below version of the plugin.
    *Version 1.9.6 - 5May18
    - Allow csv import to have spaces in column headers
    - Added smart service to convert html to csv (useful for old xls files which are actually html tables)

    It looks like there is a newer version of it available in the shared components.
    *Version 2.0.0 - 31May18
    - Thread-safe implementation of date, datetime and timestamps in the "Export SQL to flat files" and "Import CSV to Database" Smart Services

    Not sure if the issue that you are referring is being fixed but you may use the latest plugin and see if it helps.
  • 0
    Certified Lead Developer

    Can you upgrade the version of the plugin? The latest version (2.0.0) looks like that code is updated to "LOG.debug("Database name is: " + dbName);) instead of ERROR level.

    Otherwise, you could potentially add an entry to the log4j file to turn logging on this package to OFF - something like log4j.logger.com.appiancorp.ps.exceltools.util=OFF