Hello everybody.
I come to you today because I encounter some problems with my application. This app load CSV files into database. I used a thread to separate the GUI with all the work (file reading and parsing and the database interraction). Everything goes right on the test. But issues appear when we used the app with very big files (e.g. more than 2 Go files – about 10 milion lines and 50 columns CSV file). On Windows, the app crashed after a while indicating wrong argument pass to a runtime C function. I couln’d find where the problem were coming from.
I split the file into different pieces, I the importation in DB worked well. But the thing is I didn’t notice any memory leak during the importation of the very big file. The memory used by the process was stabled around 40 M according to the Windows Task Manager.
I try the same on Linux Ubuntu, and it crashed also. But I get a further error message :
terminate called after throwing an instance of ‘std::bad_alloc’
And the debugger did stop at the line where I bind a value into the QDatabaseQuery object.
So the error occured during a prepared query (INSERT INTO) with about 50 columns/parameters inseted, and after a while, so maybe after around 8 milion executions. So my question is here: does the QDatabaseQuery stores somewhere all the bindvalue? Because, I can’t see any other explanation after working a week on my code…
PS: The DBMS used is PostgreSQL
↧