enhance make

[bircni]

WIP!!!
need feedback from @silverwind as discussed

[wxiaoguang]

1. there is no other "DB" that needs to "watch".
2. there is no "ACT" at the moment.

Off-topic: maybe it should really introduce this: WIP: Pure Go SQLite3 #32628 , then no build tag is needed anymore.

[TheFox0x7]

afaik you were sort of against it - #20614 (comment)
I wouldn't mind having sqlite built-in without tags though.

[wxiaoguang]

The wasm one isn't like that CCGO based "pure Go" solution. The wasm one is 100% (at least >99%) complied from official SQLite C code, so I would consider it as safe and stable enough for production.

I was just against CCGO at that time 😄

---

To explain more for WASM ecosystem: mature compiler, mature VM. CCGO is cool and awesome, wasm is the future (IMHO)

[TheFox0x7]

My bad! I assumed this was the same solution (modernrc) and did not look at what it's imported in your PR (ncruces).
seems decent. At least better for us since we don't have to support sqlite tags.

[wxiaoguang]

Update: it seems that ncruces changed its solution.

• Old ncruces: SQLite C -> WASM compiler -> WASM code -> WASM VM
• New ncruces: SQLite C -> WASM compiler -> WASM code -> ncruces wasm2go -> Go code
• modernrc: SQLite C -> modernrc CCGO compiler -> Go code

Still, modernrc CCGO is much more complicated than ncruces wasm2go 🤣

[wxiaoguang]

cznic (native)

we call it "modernc", its package is modernc.org/sqlite

[silverwind]

modernc.org/sqlite

Right, the module path still is that: https://gitlab.com/cznic/sqlite/-/blob/master/go.mod#L1

[TheFox0x7]

ncruces seems slower to compile from scratch. Not a big issue though. It requires one less workaround but the changes in how parameters are passed seem the same.

Size diff (same base, only changed enough to get it compiling and browsable):
+6MB on ncruces
+1.1MB on modernrc

Respective branches:
https://github.com/TheFox0x7/gitea/tree/modernrc-sqlite
https://github.com/TheFox0x7/gitea/tree/ncurces-sqlite

Let me know if you have something I should try or which one to start as draft

[silverwind]

Some benchmarks at https://github.com/cvilsmeier/go-sqlite-bench, modernc seems slightly faster on average.

[wxiaoguang]

modernc +1

[TheFox0x7]

Since you're considering more targets, maybe we could have the "dev instance" one? I still have yet to figure out how can I deploy the exact instance we have preconfigured for tests which makes finding the right setup more complex than it needs to be.

[wxiaoguang]

I still have yet to figure out how can I deploy the exact instance we have preconfigured for tests which makes finding the right setup more complex than it needs to be.

Could you elaborate? Didn't get the point.

[TheFox0x7]

I worked on some bug here and I had to dive into the db schemas and repositories to find which user has which repository and what's in them to set the test up.
This is stuff which I can get in a minute from a live instance and took a lot longer because I couldn't just browse it.
The closet I got to that was to plant a breakpoint and peek at the running instance during the test but it was - as you might expect - less than ideal. There has to be a sane way to do it and either I'm unaware of it or it is not a thing yet.

I think it's useful for integration tests to see how the system is setup and for example try out your test in UI first to find the good way to perform it. Plus as a general unified demo setup it could work too.

[wxiaoguang]

Do you mean something like this ? add develop script to view and update test data #26594 ?

[silverwind]

You mean some sort of database pre-seeding for demo purposes?

[TheFox0x7]

Do you mean something like this ? add develop script to view and update test data #26594 ?

Yup. Exactly that. Though it might be out of scope for this change... now that I look at it.

You mean some sort of database pre-seeding for demo purposes?

It's less for a demo more for figuring out a setup for integration test. Demo is just a side effect here.

[wxiaoguang]

Do you mean something like this ? add develop script to view and update test data #26594 ?

Yup. Exactly that. Though it might be out of scope for this change... now that I look at it.

Maybe it's better not looking at it. 26594 made things too complicated. For "viewing test data", it can just copy the test fixtures to a temp dir, start a server, that's all. It shouldn't "update" the existing data from the "test view server".

[wxiaoguang]

Do you mean something like this ? add develop script to view and update test data #26594 ?

Yup. Exactly that. Though it might be out of scope for this change... now that I look at it.

Maybe it's better not looking at it. 26594 made things too complicated. For "viewing test data", it can just copy the test fixtures to a temp dir, start a server, that's all. It shouldn't "update" the existing data from the "test view server".

By the way, to make the CI/testing system maintainable, we should avoid adding any more test data in to the fixtures.

The tests should create their own test fixtures, or create data dynamically.

[silverwind]

The tests should create their own test fixtures, or create data dynamically.

Just model after e2e tests. every test creates all dependencies it needs itself. Fully parallelizeable, no shared fixtures, no shared state, can be made arbritrarily fast via more CPU cores. Every test is "pure".