Plugin: CSV Importer
- Maintainer: Diego Ruiz - Thomas Bayen - BXService GmbH, Krefeld
- Status: Beta
- License: GPLv2
- Sources: bitbucket
This plugin improves the normal CSV Importer process in iDempiere.
You can see the test results in the improvement section
Note: This plugin has a time improvement only for postgresql database instances
If you want to provide additional comments, please use the Discussion page
After installing the plugin login as System and go to the Report/Process window, go to the Import CSV Process record and change the classname for "de.bxservice.process.ImportCSVProcess" to run the new class.
After you do that, you can configure the Import CSV Process as usual, in the Import Template window you will see a new field (Asynchronous Commit)
In the cases you want to preserve the normal iDempiere way of doing it don't check this field, if you want the import to run faster check it and run the process.
The plugin was tested with several files, the first one a file with 5000 entries, and the behavior of the asynchronous commits was 30% faster.
In the tables below you can see the result after applying a csv file with 103.222 records. The first table shows the behavior in the normal iDempiere way, the second one the behavior after applying the plugin.
As you can see in the results the second process took 16 minutes 48 seconds less time committing, that means a 22.16% faster than the standard way.
Warning about Asynchronous Commit
As explained in the PostgreSQL documentation
Asynchronous commit introduces the risk of data loss. There is a short time window between the report of transaction completion to the client and the time that the transaction is truly committed (that is, it is guaranteed not to be lost if the server crashes). Thus asynchronous commit should not be used if the client will take external actions relying on the assumption that the transaction will be remembered. As an example, a bank would certainly not use asynchronous commit for a transaction recording an ATM's dispensing of cash. But in many scenarios, such as event logging, there is no need for a strong guarantee of this kind.
- Plugin:_RED1_NINJA that has automated conversion of attached CSV into a new model fully rendered in Menu tree with CSV content injected as data.