Weekly Status Log: Query Interfaces

[domoritz]

This is for uwdata#483, uwdata#510, and uwdata#506. Code is in https://github.com/cmudig/mosaic/tree/prepare.

[dorianpan]

September 20, 2024

– should try a different way of extracting the params (visitor pattern as a ref) different from the one in toString()
– server implementation (python first)
– possibly try to refactor toString() into toSql() and have two versions for prepared and non-prepared

Goal next week:
– layout implementation plan for adding the prepare statement
– have an initial implementation based on the plan

[dorianpan]

September 25th, 2024

Discussion today
– rename toString() function
– safety issue with current implementation -- ensure the approach is robust
– use a helper function probably?
– better logic for adding “?”
https://docs.google.com/document/d/1RiO9qCwJS333HMGcDEAKb7xV80oZwGfYoFzIyARqeZg/edit

[dorianpan]

September 27th, 2024

Discussion today:
9.27 meeting
— front end sends the generic string & param list (still buggy but should be done soon)
— server almost the same as the old implementation (already passed the test)
— Dorian’s next step:

• Make sure the toSql() function works
• Test it with current backend implementation
  — Plan to be completed (doug)
• Node connector
• Update json format in data-server.js
• Add prepared config param
• arrowBuffer (DuckDBStatement): this.statement instead of this.con?
• Add tests
• cache: update cache to include prepared params, etc, update calls to cacheKey
• prepare(sql): add map to check for prepared
• What is bundle?
• Python connector
• What are handlers doing? (what is app?)

[domoritz]

What is bundle?

Bundles are prepared sets of queries that can be loaded into the cache. There is a demo app in https://github.com/uwdata/mosaic/blob/main/dev/bundle.html.

[dorianpan]

October 2, 2024

Douglas: updated the node server code to accommodate for the update on the sql builder; started python server update

-- todo: update python server code to prepare query in the sample script format

Dorian: the toSql function is able to return an object containing the param array and the query string with ? marks

-- todo: need to support both prepared and non-prepared versions by having an indicator variable as an input

Next step: test the implementation together & make changes if anything doesn't work

[dorianpan]

October 4, 2024

Pushed all changes to server and sql builder to remote branch prepare
Modified test cases for Python server so now able to run test locally

Pending: need to find a way to connect to the python server and run test with both sql builder and the python server; maybe write a script that make requests to localhost?

[dorianpan]

October 9, 2024

meeting updates
Douglas -- updated python and node server so now they are able to handle prepared statements

Dorian -- implemented the query generator to support generating both prepared and non-prepared query strings (buggy now needs a closer look)

goals next week:
-- figure out if prepared statement is per session
-- able to enable & disable prepared statement (mostly client side)
-- create unit tests and start to think about how to modify the interface so there will be an option to select whether the statement is prepared or not