64bit/Database/Stores
goal: to have downloading and installing each music store database be very fast
current design:
- store songs are marked as such, that way when we load the database, if the user has "turned off" a particular store, we can just purge all song entries associated with that store
- kjams downloads an xml or json file for a particular store
- kjams parses each song in the downloaded.
- for each song, create a brand new song ID, start a transaction, then parse each bit of metadata (table colums) and adds each to the DB as an individual call to sql, then when the song is completed, checkpoint the transaction. this is an extremely slow process
- this may be happening for up to N stores at once, each on a different thread, and gets slower as more stores are parsed at once due to database locking
- when entire store has completed parsing, we must then associate each song with an existing song (swap the newly created ID of the song with the existing song ID, that way any playlists that already have that song in it will now reference the new song with possibly updated meta data. after all songs are done, purge all NEW song IDs (since they were swapped with old song IDs, we're actually purging the old songs, keeping the new ones with the old IDs
new design:
- do steps as above, except in step 2, first see if sqlite file is available for download, if so, download THAT instead of xml / json, then skip to step 3 below
- before step 4 above, the song data is stored in a new, separate database file, unreleated to the existing DB
- before step 6 above, skip to step 4 below
- kJams loads the store sqlite file and merges it into the existing Library.
- note the SongID stored per song in the store is ONLY unique within each store, and may conflict with SongID of Library once loaded: while loading we must give each record a new ID