Moving data from QVD to SQL Server

I had to move some large data (40GB+) from QVD file format (QlikView/Qlik Sense), into a database (SQL Server). I discuss some of the pitfalls I encountered in this post…

Filter Early

My QVD had data I didn’t need;

Store QVD Into CSV

Use Qlik Sense load script to store each of the relevant QVD files (one “table” per QVD), into a separate CSV file.

CSV will not contain “schema” data, so to prepare for the next step, do this:

SQL Server Import Wizard

Choose datatypes for the CSV “Input Columns”

Allow the Input Columns to determine the resulting Output Columns in a new table (just an easy way to get your table created for you, automatically)

JOIN in SQL Server

I use a JOIN in SQL Server (see earlier comment, I ran into limits trying to JOIN directly in the Qlik Sense load script)

Simplify the Key

In my case, I was joining a large table (“Main”) to a small lookup table (“Lookup”) via a “bridge” table (“Bridge”). The key to join the “Main” table to the “Bridge” (“MainBridgeKey”) was at a very low-level of detail; and the resulting JOIN increases rowcount from 300K -> 300M

I used HASHBYTES function to reduce the “Main” table into a “SimplifiedMain”