Initial draft at SQLite Backend

[cddr]

No description provided.

[dgrnbrg]

Could we move this down to the creation of the SQLiteBackend? I think that would make it easier to review by not changing the indent, and that's the only time we need the DB to be created.

Also, what are your thoughts on using memory vs. disk-backed DB. I feel like backing with disk is more interesting to trigger the IO behavior we might encounter.

[cddr]

Yeah memory was nice while getting started so I didn't have to worry about resetting the schema.

I didn't actually mean to submit a PR so early but thanks for your comments. I'll refactor the db lifecycle. This was just the easiest way to see some quick results. I'm not sure how to interpret that spreadsheet though.

[dgrnbrg]

I like how you managed to get refcounting into SQLite, but I'm worried that the refcounting IO overhead might cause issues. On the other hand, maybe it won't. Have you ever profiled a JVM application? I'd be happy to teach you how to do so--the benchmarking tool makes it really easy.

[cddr]

Not something low-level like this. Would welcome the tutoring :-)

[dgrnbrg]

Great! Once you're able to run the benchmark, let's jump on a hangout and I'll walk you through some of the techniques I use to identify perf bottlenecks--we'll look for operations that use too much CPU, too much I/O, or cause weird cross-interferences :). You should have my google account from invite I requested on drive.

[dgrnbrg]

This is awesome. I think this is pretty much ready to merge, except for one thing: could you write a bit about the schema/structure of the data in SQLite, so that we can understand how the data is encoded?

[cddr]

Cool!

Sorry for the extra changes but I also took the liberty of renaming the namespace to be "hitchhiker.jdbc" because there's nothing specific to sqlite in there so it should just work with any database you can find a jdbc driver for.

There might be one other thing which to surface the selection of a backend to the user through outboard/create but I can do that in a new PR. I was thinking of adding a multi-method that dispatches off the name passed to outboard/create and returns a "backend constructor" and adding a slot in OutboardConnection to hold it.

[dgrnbrg]

That looks great!

For Outboard, I had originally envisioned it as being a persistent in-memory backed data structure. Now that you bring it up, I can definitely see the value in extending the API to support plugging in alternate backends. Maybe the create function could take the backend as an argument, and then we could have namespaced functions like outboard.redis/create and outboard.jdbc/create for parametricity?

[cddr]

Maybe the create function could take the backend as an argument

That would also work but I thought create was supposed to be analogous to datomic's create-database (or would you want to avoid making the API exactly the same as datomic).

[dgrnbrg]

Outboard != datomic, API-wise. Outboard's a proof-of-concept API to make the HH trees usable by end-users. There will be a different API that mirrors the Datomic API.