Benjamin Dauvergne
a08c629f75
feeder: create an unique slug if necessary ( #42911 )
2021-08-19 11:59:03 +02:00
Benjamin Dauvergne
05c4031776
feeder: prevent situation of half-dropped schema ( #54658 )
...
To prevent loosing currently loaded data wcs-olap, failing ro rename the
temporary schema to its final name, wcs-olap will:
- first, inside a transaction, rename the current schema instead of
dropping it, then rename the new schema to the current schema's name;
in case of failure it will retry 33 times sleeping 1 second between
each attempt;
- if successfull, drop the renamed old schema, again in a retry loop, if
it fails to drop it logs an error, without aborting the current
feeding.
2021-08-17 23:07:54 +02:00
Benjamin Dauvergne
e0a28c2f85
feeder: set synchronous_commit to off during feed ( #56039 )
2021-08-17 19:27:48 +02:00
Benjamin Dauvergne
47a85d8531
feeder: add option post-sync-commands ( #56164 )
...
Exemple:
[wcs-olap]
post-sync-commands = ALTER DEFAULT PRIVILEGES IN SCHEMA {schema} GRANT SELECT ON TABLES TO user;
GRANT USAGE ON SCHEMA {schema} TO user;
GRANT SELECT ON ALL TABLES IN SCHEMA {schema} TO user;
Those commands will be launched after the synchronization, interpolation
variables are usable (like {schema} for the target schema name).
2021-08-14 23:41:16 +02:00
Benjamin Dauvergne
5baffc8156
cmd: use Logger.setLevel() ( #56038 )
...
Directly setting logger.level should never used as it does not clear the
isEnabledFor cache of the logger which breaks some tests.
2021-08-14 23:35:48 +02:00
Benjamin Dauvergne
7690321265
tests: add test_sentry ( #56038 )
2021-08-14 10:54:16 +02:00
Benjamin Dauvergne
9476ad799b
misc: integrate with sentry ( #56038 )
...
To use it add the following lines to /etc/wcs-olap/config.ini:
[sentry]
dsn = https://apikey@sentry.example.com/1
environment = prod
2021-08-07 15:35:30 +02:00
Frédéric Péters
48b8adaf27
tox: run tests against django 2.2
2021-07-03 14:47:24 +02:00
Frédéric Péters
61c6085fae
tox: limit psycopg2 to < 2.9 ( #54925 )
2021-07-03 14:46:45 +02:00
Benjamin Dauvergne
b3a59edb54
Revert "feeder: set deadlock_timeout to 10 seconds ( #54658 )"
...
This reverts commit c74766bf2c
.
2021-06-14 09:44:41 +02:00
Frédéric Péters
9bd160e0dd
do not store digits value if outside integer range ( #54702 )
2021-06-11 08:51:45 +02:00
Benjamin Dauvergne
f8c4a03464
tests: add invalid integer to wcs fixture ( #54702 )
2021-06-11 07:41:29 +02:00
Benjamin Dauvergne
c74766bf2c
feeder: set deadlock_timeout to 10 seconds ( #54658 )
...
It makes wcs-olap always win in a deadlock with another client.
2021-06-08 17:26:43 +02:00
Benjamin Dauvergne
ee4bcadcc0
feeder: log failing SQL statements ( #45314 )
2021-05-28 08:19:34 +02:00
Benjamin Dauvergne
d44fafb844
misc: handle invisible statuses using real_status ( #40643 )
...
The formdata API returns the visible status in data.workflow.status,
if we want to index the current status we must use
data.workflow.real_status.
2021-05-18 19:50:11 +02:00
Benjamin Dauvergne
f7f1a6d26f
feeder: filter empty formdefs as early as possible ( #52015 )
2021-04-01 13:19:20 +02:00
Frédéric Péters
1640abbfdc
add index on dates ( #51681 )
2021-03-16 17:20:01 +01:00
Frédéric Péters
1dc7fb5833
trivial: remove debugging print
2021-03-13 11:05:51 +01:00
Frédéric Péters
30e56df41a
misc: add social network submission channel ( #51816 )
2021-03-09 10:35:18 +01:00
Benjamin Dauvergne
dc36901dde
feeder: store truncated temp schema's name ( #50835 )
2021-02-04 10:35:24 +01:00
Benjamin Dauvergne
1d35d39f6f
misc: remove SQL objects in order to use less locks ( #43108 )
2021-02-03 18:05:25 +01:00
Benjamin Dauvergne
fe2f5b5d89
do not use count to compute emptyness of formdef ( #50368 )
2021-01-21 11:37:46 +01:00
Serghei Mihai
30f6873d27
feeder: write new model file after schema renaming ( #42320 )
gitea/wcs-olap/pipeline/head There was a failure building this commit
Details
2021-01-11 14:28:51 +01:00
Frédéric Péters
cbd0ec80a6
build: update to use origin/main
2020-12-26 15:21:15 +01:00
Benjamin Dauvergne
ea72fe5d7e
use truncate_pg_identifier on fields and table names ( #48684 )
2020-12-17 14:41:25 +01:00
Serghei Mihai
fc92793e49
feeder: try to cast digit string fields with numeric value only ( #48426 )
2020-11-26 10:37:47 +01:00
Benjamin Dauvergne
a16fb4ee98
misc: accept duplicate fields with the same type ( #42429 )
2020-09-25 20:39:41 +02:00
Frédéric Péters
326f53e84a
trivial: use https to download wcs for tests (via #46904#note-2)
2020-09-24 19:57:39 +02:00
Valentin Deniaud
483dfb3089
tox: tell setuptools to use distutils from stdlib ( #46252 )
2020-09-01 14:26:25 +02:00
Nicolas Roche
3eb714f93a
wcs_api: provide new include-disabled parameter ( #45898 )
2020-08-31 17:24:06 +02:00
Benjamin Dauvergne
dd9e234dde
Jenkinsfile: use python3-venv
2020-08-31 11:02:21 +02:00
Frédéric Péters
156491657d
tox: limit quixote to <3.2, to keep tests working with python 3.5
2020-08-11 08:37:41 +02:00
Benjamin Dauvergne
9f5ff07675
wcs_api: use WcsApi.batch_size for FormDatas batch size ( #44970 )
2020-07-09 14:26:43 +02:00
Benjamin Dauvergne
909712e15a
feeder: remove import six ( #43546 )
2020-06-26 11:42:54 +02:00
Benjamin Dauvergne
afb70fe986
feeder: create measure for integer fields ( #43546 )
2020-06-26 11:42:54 +02:00
Benjamin Dauvergne
e4e8779661
tests: add a string column with digits validation ( #43546 )
2020-06-26 11:42:50 +02:00
Benjamin Dauvergne
af2513e56e
jenkins: adopt authentic way of doing tox.ini and Jenkinsfile
2020-06-26 11:40:21 +02:00
Benjamin Dauvergne
6e4cef6d5c
feeder: update public.dates atomically ( #44244 )
2020-06-26 11:38:42 +02:00
Benjamin Dauvergne
579a118a5f
tests: test schema name length check
2020-06-03 10:07:42 +02:00
Benjamin Dauvergne
d9409b15bd
misc: truncate schema_temp to 63 characters ( #43165 )
2020-05-20 17:37:10 +02:00
Benjamin Dauvergne
44d11e2b02
misc: raise ValueError is schema name is too long ( #43165 )
2020-05-20 17:30:49 +02:00
Benjamin Dauvergne
f45373eb51
misc: quote table names when droping ( #43063 )
2020-05-19 10:00:31 +02:00
Frédéric Péters
f3ed695059
tests: adapt to changes in wcs runscript command ( #42098 )
2020-05-15 10:57:05 +02:00
Serghei Mihai
799d799565
feeder: fix debug message on schema renaming ( #42323 )
2020-04-30 16:42:41 +02:00
Serghei Mihai
c68304bf96
feeder: drop old schema sequentially ( #42322 )
2020-04-30 16:42:41 +02:00
Paul Marillonnet
31540a886f
debian: adapt dependency overriding to python3 ( #41303 )
2020-04-03 11:17:24 +02:00
Serghei Mihai
d11d196fa8
feeder: preseve categories and form names order ( #36930 )
2020-03-02 22:07:03 +01:00
Benjamin Dauvergne
3afc79b42e
wcs_api: handle empty dict in FormData.workflow ( #39373 )
2020-01-31 18:26:21 +01:00
Benjamin Dauvergne
8fc092f4d7
feeder: add prefix to join's names of fields ( #39373 )
2020-01-31 17:51:12 +01:00
Benjamin Dauvergne
04ad4ee2d2
feeder: factorize values in field's schemas ( #39373 )
2020-01-31 17:51:12 +01:00