entrouvert
/
wcs-olap
16
0
Fork 0
Code Issues Pull Requests Releases Activity
202 Commits 6 Branches 81 Tags 385 KiB
Commit Graph

202 Commits

Author SHA1 Message Date
Frédéric Péters 88cf168b10 only consider bool/item/items fields if explicitely marked for stats (#63432) 2022-04-01 09:48:00 +02:00
Benjamin Dauvergne 420c724be7 feeder: ignore non string items (#60243) 2022-01-21 11:07:26 +01:00
Emmanuel Cazenave 016ff7e1ee jenkins: show execution context in coverage reports (#60446) 2022-01-11 16:18:00 +01:00
Frédéric Péters 70dc277748 jenkins: use python3 to get a port for postgresql (#60467) 2022-01-10 17:33:35 +01:00
Benjamin Dauvergne 642b7efa3e feeder: create UNLOGGED tables (#59656) 2021-12-22 10:56:22 +01:00
Frédéric Péters f94da4a239 tox: add unidecode somehow now missing 2021-12-17 17:40:43 +01:00
Frédéric Péters 5536b0dd07 tox: declare new wcs dependency (dnspython) 2021-12-15 08:19:40 +01:00
Frédéric Péters 817d5a789b jenkins: build packages for buster & bullseye 2021-12-12 11:38:08 +01:00
Frédéric Péters 7aaa76feb8 tox: add bleach, new w.c.s. dependency 2021-11-30 17:26:20 +01:00
Frédéric Péters c7e1fd583e debian: switch to debhelper-compat 12 (#57538) 2021-10-10 10:53:29 +02:00
Frédéric Péters cc28626b9b debian: do not run tests 2021-10-10 10:53:29 +02:00
Benjamin Dauvergne a08c629f75 feeder: create an unique slug if necessary (#42911) 2021-08-19 11:59:03 +02:00
Benjamin Dauvergne 05c4031776 feeder: prevent situation of half-dropped schema (#54658) 
To prevent loosing currently loaded data wcs-olap, failing ro rename the
temporary schema to its final name, wcs-olap will:

- first, inside a transaction, rename the current schema instead of
  dropping it, then rename the new schema to the current schema's name;
  in case of failure it will retry 33 times sleeping 1 second between
  each attempt;

- if successfull, drop the renamed old schema, again in a retry loop, if
  it fails to drop it logs an error, without aborting the current
  feeding.
 2021-08-17 23:07:54 +02:00
Benjamin Dauvergne e0a28c2f85 feeder: set synchronous_commit to off during feed (#56039) 2021-08-17 19:27:48 +02:00
Benjamin Dauvergne 47a85d8531 feeder: add option post-sync-commands (#56164) 
Exemple:

  [wcs-olap]
  post-sync-commands = ALTER DEFAULT PRIVILEGES IN SCHEMA {schema} GRANT SELECT ON TABLES TO user;
    GRANT USAGE ON SCHEMA {schema} TO user;
    GRANT SELECT ON ALL TABLES IN SCHEMA {schema}  TO user;

Those commands will be launched after the synchronization, interpolation
variables are usable (like {schema} for the target schema name).
 2021-08-14 23:41:16 +02:00
Benjamin Dauvergne 5baffc8156 cmd: use Logger.setLevel() (#56038) 
Directly setting logger.level should never used as it does not clear the
isEnabledFor cache of the logger which breaks some tests.
 2021-08-14 23:35:48 +02:00
Benjamin Dauvergne 7690321265 tests: add test_sentry (#56038) 2021-08-14 10:54:16 +02:00
Benjamin Dauvergne 9476ad799b misc: integrate with sentry (#56038) 
To use it add the following lines to /etc/wcs-olap/config.ini:

[sentry]
dsn = https://apikey@sentry.example.com/1
environment = prod
 2021-08-07 15:35:30 +02:00
Frédéric Péters 48b8adaf27 tox: run tests against django 2.2 2021-07-03 14:47:24 +02:00
Frédéric Péters 61c6085fae tox: limit psycopg2 to < 2.9 (#54925) 2021-07-03 14:46:45 +02:00
Benjamin Dauvergne b3a59edb54 Revert "feeder: set deadlock_timeout to 10 seconds (#54658)" 
This reverts commit c74766bf2c.
 2021-06-14 09:44:41 +02:00
Frédéric Péters 9bd160e0dd do not store digits value if outside integer range (#54702) 2021-06-11 08:51:45 +02:00
Benjamin Dauvergne f8c4a03464 tests: add invalid integer to wcs fixture (#54702) 2021-06-11 07:41:29 +02:00
Benjamin Dauvergne c74766bf2c feeder: set deadlock_timeout to 10 seconds (#54658) 
It makes wcs-olap always win in a deadlock with another client.
 2021-06-08 17:26:43 +02:00
Benjamin Dauvergne ee4bcadcc0 feeder: log failing SQL statements (#45314) 2021-05-28 08:19:34 +02:00
Benjamin Dauvergne d44fafb844 misc: handle invisible statuses using real_status (#40643) 
The formdata API returns the visible status in data.workflow.status,
if we want to index the current status we must use
data.workflow.real_status.
 2021-05-18 19:50:11 +02:00
Benjamin Dauvergne f7f1a6d26f feeder: filter empty formdefs as early as possible (#52015) 2021-04-01 13:19:20 +02:00
Frédéric Péters 1640abbfdc add index on dates (#51681) 2021-03-16 17:20:01 +01:00
Frédéric Péters 1dc7fb5833 trivial: remove debugging print 2021-03-13 11:05:51 +01:00
Frédéric Péters 30e56df41a misc: add social network submission channel (#51816) 2021-03-09 10:35:18 +01:00
Benjamin Dauvergne dc36901dde feeder: store truncated temp schema's name (#50835) 2021-02-04 10:35:24 +01:00
Benjamin Dauvergne 1d35d39f6f misc: remove SQL objects in order to use less locks (#43108) 2021-02-03 18:05:25 +01:00
Benjamin Dauvergne fe2f5b5d89 do not use count to compute emptyness of formdef (#50368) 2021-01-21 11:37:46 +01:00
Serghei Mihai 30f6873d27 feeder: write new model file after schema renaming (#42320)
gitea/wcs-olap/pipeline/head There was a failure building this commit Details
2021-01-11 14:28:51 +01:00
Frédéric Péters cbd0ec80a6 build: update to use origin/main 2020-12-26 15:21:15 +01:00
Benjamin Dauvergne ea72fe5d7e use truncate_pg_identifier on fields and table names (#48684) 2020-12-17 14:41:25 +01:00
Serghei Mihai fc92793e49 feeder: try to cast digit string fields with numeric value only (#48426) 2020-11-26 10:37:47 +01:00
Benjamin Dauvergne a16fb4ee98 misc: accept duplicate fields with the same type (#42429) 2020-09-25 20:39:41 +02:00
Frédéric Péters 326f53e84a trivial: use https to download wcs for tests (via #46904#note-2) 2020-09-24 19:57:39 +02:00
Valentin Deniaud 483dfb3089 tox: tell setuptools to use distutils from stdlib (#46252) 2020-09-01 14:26:25 +02:00
Nicolas Roche 3eb714f93a wcs_api: provide new include-disabled parameter (#45898) 2020-08-31 17:24:06 +02:00
Benjamin Dauvergne dd9e234dde Jenkinsfile: use python3-venv 2020-08-31 11:02:21 +02:00
Frédéric Péters 156491657d tox: limit quixote to <3.2, to keep tests working with python 3.5 2020-08-11 08:37:41 +02:00
Benjamin Dauvergne 9f5ff07675 wcs_api: use WcsApi.batch_size for FormDatas batch size (#44970) 2020-07-09 14:26:43 +02:00
Benjamin Dauvergne 909712e15a feeder: remove import six (#43546) 2020-06-26 11:42:54 +02:00
Benjamin Dauvergne afb70fe986 feeder: create measure for integer fields (#43546) 2020-06-26 11:42:54 +02:00
Benjamin Dauvergne e4e8779661 tests: add a string column with digits validation (#43546) 2020-06-26 11:42:50 +02:00
Benjamin Dauvergne af2513e56e jenkins: adopt authentic way of doing tox.ini and Jenkinsfile 2020-06-26 11:40:21 +02:00
Benjamin Dauvergne 6e4cef6d5c feeder: update public.dates atomically (#44244) 2020-06-26 11:38:42 +02:00
Benjamin Dauvergne 579a118a5f tests: test schema name length check 2020-06-03 10:07:42 +02:00