Commit Graph

51 Commits

Author SHA1 Message Date
Benjamin Dauvergne 46f56b3808 misc: move AddIndex to end of 0002 migration (#39544)
Automatic generation does not sort AddIndex for manual indexes correctly
with respect to Remove/AddField.
2020-04-15 11:02:53 +02:00
Benjamin Dauvergne c8de0807ac misc: add proper 0002 migration (#39544)
0001 should never be modified apart from cosmetic changes.
2020-04-15 10:47:26 +02:00
Benjamin Dauvergne d02c2dc884 misc: remove explicit byte string in migrations for py3 (#39544) 2020-04-15 10:40:53 +02:00
Benjamin Dauvergne 8cd865d3f9 misc: fix package path in migration (#39544) 2020-04-15 10:40:36 +02:00
Benjamin Dauvergne da8822e821 misc: restore initial 0001 migration (#39544) 2020-04-15 10:31:37 +02:00
Emmanuel Cazenave 60d2884f8c use items() instead of iteritems() (#39533) 2020-02-04 17:27:47 +01:00
Emmanuel Cazenave 253fc22a15 adapt chunk read to binary data (#39533) 2020-02-04 17:27:19 +01:00
Emmanuel Cazenave 436022883a open file in binary mode (#39533) 2020-02-04 17:26:51 +01:00
Emmanuel Cazenave 5747367a56 use response.text when expecting text (#39533) 2020-02-04 17:26:18 +01:00
Emmanuel Cazenave 0ecaa0d858 grab url methods form six (#39533) 2020-02-04 17:25:01 +01:00
Emmanuel Cazenave 87c7468ab1 catch binascii.Error when decoding base64 (#39533) 2020-02-04 17:23:20 +01:00
Emmanuel Cazenave 5b788a5dff grab HTMLParser from six and adapt initialization to py3 (#39533) 2020-02-04 17:21:41 +01:00
Emmanuel Cazenave 37534c6980 use six types when calling isinstance() (#39533) 2020-02-04 17:19:16 +01:00
Emmanuel Cazenave 90bdb52e90 use python_2_unicode_compatible for objects representation (#39533) 2020-02-04 17:13:50 +01:00
Emmanuel Cazenave 090e4d2bc3 use force_text() instead of unicode() (#39533) 2020-02-04 17:09:09 +01:00
Emmanuel Cazenave 671afb17ec migration: use text instead of bytes (#39533) 2020-02-04 17:01:46 +01:00
Michael Bideau a0ae081b74 Models: fixed url concatenation (removed '/' at the begining of the second part) 2019-12-05 21:31:40 +01:00
Michael Bideau 05a81aa993 Pylint: fixed 'keyword-arg-before-vararg' and module order issues 2019-10-29 18:26:56 +01:00
Michael Bideau 1214526b1a Replaced courrier_type 'decision_non_opposition_DP' with 'decision_nom_opposition_sr' 2019-10-28 18:38:45 +01:00
Michael Bideau 6809c75d33 Make the connector more generic; Fixed urls and some pylint issues
Features
    * Added 'get_courrier_type' endpoint that return a 'courrier type' for a given type_dossier
    * Added url parameter 'type_dossier_detaille' to 'create_dossier' to post it in the request body
    * Replaced the way 'type fichier' is defined: now looks into 'XXX_type_raw' for file 'XXX'

Fixes
    * Url components are now encoded
    * Pylint warnings
    * Loging do not raise an error when 'files' var is empty in 'create_dossier'

Tests
    * Added 2 new errors tests to 'test_create_dossier'
    * Added 'test_get_courrier_type' to test endpoint 'get_courrier_type'
    * Fixed pylint warnings
2019-10-24 17:24:55 +02:00
Frédéric Péters 6197d8ae62 misc: fix debug log call to receive string as first parameter 2019-10-16 08:37:12 +02:00
Michael Bideau 38061d4593 Locales: added translations for 'fr' locale 2019-08-22 16:41:51 +00:00
Michael Bideau a739371cef Features: adding verbose name to all fields (mainly ForwardFile ones) 2019-08-22 16:41:28 +00:00
Michael Bideau a951f062f0 Refactoring: fix most of the (many) 'flake8' complaints 2019-08-22 16:02:06 +00:00
Michael Bideau afcca49f53 Refactoring: fix most of the (many many) 'pylint' complaints 2019-08-22 14:44:52 +00:00
Michael Bideau 2786bc29e7 License: prepended license text to most files 2019-08-21 15:02:33 +00:00
Michael Bideau 8474372bef Locales: cleaning extra 'fr' PO file (should not have been commited previously) 2019-08-21 14:39:57 +00:00
Michael Bideau 35fe3e7411 Setup.py: added locales compilation; Locales: added 'fr' PO file; Fixes: semantics and typos 2019-08-21 14:22:35 +00:00
Michael Bideau 990f1fb7bc Added 'guichet' (planned availability), added CRUD views for each entity, many fixes and refactoring
Features:
    * A connector now can be associated with 'collectivites',
      and each of them can have one 'guichet'.
      A 'collectivite' have a name and an openADS technical identifier
      A 'guichet' allow to define hours and days for when its 'collectivite' is "open"
    * Each of those entities have a view to CRUD it

    * For entity ForwardFile:
       - Added association with a connector, and eventually a 'collectivite'
       - Added ForwardFiles 'size' field
       - Added updating fields 'content_type', 'file_hash' and 'size' on save()
       - Added custom validation on save()

    * For entity AtrealOpenads:
       - Added permissions on each endpoint
       - Added 'email' field support for each type of 'demandeur'
       - 'numero_dossier' is now required in the url for some endpoints (previously was a GET param)
       - Added 'upload_user_files' as an endpoint (but can still be used as a method with request=None)
       - The 'upload_user_files()' method now only handle forward file that have status='pending'

Fixes:
    * Added "*args" and "**kwargs" arguments to some connector endpoint methods
    * Added 'verbose_name' and 'ordering' on each entity META
    * Added decorator '@force_encoded_string_output' to prevent utf-8 issues with python2
    * Added __repr__, __str__ and __unicode__ functions to each entities
    * Added database indexes for each entity
    * Commented out useless JSON schema imports
    * Removed unused variable assignations

Refactoring:
    * Moved utilities functions to utils.py file
    * Added a BaseModel to provide some default functions for Models
    * Added enum/translations of hard-coded values for ForwardFile 'upload_status' field

Tests:
    * Added test for each entity
    * Added tests files specific to utilities, forms and views
    * Total code coverage is 99% with only 10 statement missed
2019-08-20 17:00:12 +00:00
Michael Bideau e8de2e66e8 JSON schemas: removed __main__ part 2019-08-20 16:23:42 +00:00
Michael Bideau 73669b0f33 JSON schemas: fixed a typo ('Un particulier' instead of 'Une personne physique') 2019-08-19 16:43:25 +00:00
Michael Bideau 4fc61d5f5e Added 'lettre_type' param to 'get_courrier()' endpoint 2019-07-29 14:35:56 +02:00
Michael Bideau d4080dae85 Added 'collectivite' param to 'create_dossier()' endpoint, removed useless '*args' and '**kwargs' 2019-07-29 14:30:50 +02:00
Michael Bideau 98cd089254 100% test coverage, dict strings are truncated when dumped, error messages homogeneized, and more
Features:
    * replaced the loging of json payload with the use of a DictDumper that lazy truncate strings too long
    * homogenize error messages

Fixes:
    * changed the signature of 'check_file_dict()' to match other functions and fix a bug
    * changed the signature of 'upload2ForwardFile()' to force input for 'type_fichier' argument

Refactoring:
    * removed dead code in 'upload_user_files()'
    * using 'basestring' instead of testing for 'str' and 'unicode'
    * renamed 'requerant' by 'demandeur'
    * removed useless and maybe problematic 'join()' (forgotten from the previous commit)
    * added more comments

Tests:
    * 100% test coverage!
    * fixed failing tests
    * added tests for every function
    * completed missing tests (mostly for exceptions)
    * added more comments
    * added the lines of missing statement in the coverage report
2019-07-25 14:29:06 +02:00
Michael Bideau 571316fcf4 JSON Schema input validation enabled (again), utf-8 decode error fixed, and more
Feature:
    * JSON Schema input validation enabled (again)

Fixes:
    * utf-8 decode error fixed: added unicode to litterals
    * removed useless and maybe problematic 'str()' forced casting
    * remove useless and maybe problematic 'join()' on list of file ids

Refactoring:
    * property 'extra_debug_enabled' removed (not used since some commits)
    * in function 'normalize()' only cast to 'unicode' if value is not already 'unicode'
2019-07-23 15:51:29 +02:00
Michael Bideau 6e709a0448 JSON Schema fixed, commented and updated with 'qualite' conditions on 'denomination' and 'raison_sociale' 2019-07-23 15:44:13 +02:00
Michael Bideau 6b4106ae25 Removed JSON schema validation, Added support for 'mandataire' and 'pouvoir_mandat', fixed bugs 2019-07-22 20:21:09 +02:00
Michael Bideau 719687287c Fix: UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 11: ordinal not in range(128) 2019-07-22 17:41:00 +02:00
Michael Bideau 62dae769d6 Refactoring: remove 'extra_debug' field and replaced all 'self.debug' calls by 'self.logger.debug' 2019-07-18 22:54:50 +02:00
Michael Bideau 02f4a071fc Use HTTPResource, JSON schema for inputs, JSON lazy logging, added ForwardFile.attempt, and more
Features:
    * class AtrealOpenads now extends HTTPResource for HTTP Basic credentials
    * check_status is now also an endpoint (endpoint 'test_openads_connexion' deleted)
    * added JSON schema to validate POST request input (ouput schemas are defined too but unused for now)
    * added class 'LogJsonPayloadWithFileContent' to lazy log a json payload with content file filtered
    * added 'attempt' field to class 'ForwardFile' that count the attempt of uploading to openADS.API
    * added more controls over response data received

Refactoring:
    * removed the file content in function 'get_fwd_files_status()' and added content size instead
    * splitted the function 'get_fwd_files_status()' in two: one that retrieve detailed files, one only the summary
    * added function 'log_json_payload()' to help to log json payloads
    * added 2 functions to factorize the process of getting files from json payload
    * added function 'check_file_dict()' to factorize the process of checking a file dictionary

Fixes:
    * Default filename are now under 50 chars
    * Make the function 'get_fwd_files_status()' consistent in its return

Tests:
    * updated the test to reflect the use of HTTPResource and the split of 'get_fwd_files_status()'

More:
    * Added a lot of comments
    * Added endpoint anotation to validate response body, but its commented as the 'get' helper is not defined (not like the 'post' one)
2019-07-18 21:47:38 +02:00
Michael Bideau 8593779954 Added testing with tox, added AGPLv3 license, replaced namespace, and more
Packaging:
    * added author, description, classifiers, scripts and requirements in 'setup.py'

Tests:
    * added support for 'tox.ini'
    * moved 'tests' directory outside main python module

Fixes:
    * namespace is 'atreal_openads' instead of 'passerelle.apps.atreal_openads'
    * increased timeout removed from 'get' requests
    * fixed timeout typo on 'self.openADS_API_timeout'
2019-07-16 11:27:19 +02:00
Michael Bideau 18e57caf99 Features: added a 1h timeout to every requests 2019-07-15 13:14:38 +02:00
Michael Bideau ee0173698e Added 'test_openads_connexion' endpoint and README completed 2019-07-11 18:57:07 +02:00
Michael Bideau 522634d564 Added 'extra_debug' field and a function to only log debug message if extra_debug is enabled 2019-07-11 18:22:47 +02:00
Michael Bideau 7020d3c2d0 Cleanup: removed useless endpoints and test cases 2019-07-11 17:41:38 +02:00
Michael Bideau 6fe6307506 Tests: removed useless files added by mistake and old 'tests' directory 2019-07-11 17:18:08 +02:00
Michael Bideau 1ac8c2f14e Making tests independant of 'passerelle' and 'vagrant', added HTTP Basic credentials
Tests:
    * moved 'tests' folder inside 'atreal_openads' module
    * removed useless settings.py file
    * removed useless imports
    * added test file 'resources'
    * fixed file path (no more vagrant)
    * updated the test case of 'upload2ForwardFile' to test for 'plan' type_fichier
    * added HTTP Basic credentials to 'atreal_openads' fixture (removed 'token')

Fixes:
    * stop guessing ForwardFile.type_fichier based on file path
2019-07-11 17:07:41 +02:00
Michael Bideau ce44b0a733 Added HTTP Basic authentication, ForwardFiles content deleted on success, plus logging and cleaning
Features:
    * added HTTP Basic authentication (+ login/password fields, - token field)
    * ForwardFiles content deleted on upload success (object/status kept)

Logging:
    * added debuging and warning messages
    * replaced the python interpolation '%' by function args ','

Cleaning:
    * removed functions 'ajob', 'afile'
    * removed unused imports

Fixes:
    * function 'get_upload_path()' returns unique names
    * added missing imports
    * using string in dictionary keys
2019-07-10 18:23:31 +02:00
Michael Bideau 3c53170e4a Supporting to create a 'dossier' from a real request (no more hardcoded payload)
Features:
    * added support for a payload build from the request instead of hardcoded in
      function 'create_dossier()'
    * added a warning message when an APIError is raised all functions
    * added the name/key of the error in the error message build from a response
    * added a function 'normalize()' to normalize user's input in the payload
    * changed endpoint arguments coming from request parameters instead of URI path
      in functions 'create_dossier()', 'get_dossier()', 'get_courrier()' and
      'get_fwd_files_status()'
    * changed JSON response to use an explicit key to hold file data in function
      'create_dossier()':'recepisse' and 'get_courrier()':'courrier'
    * removed the useless key 'data' in all the JSON response
    * removed useless case 'jsondata' in function 'afile()'

Cleaning:
    * Removed useless variable 'dossier_payload'

Tests:
    * added request containing the payload in the test case for 'create_dossier()'
    * added the name/key of the error in the error message build from a response
    * used the explicit keys 'recepisse' and 'courrier' in JSON responses
    * removed the 'data' key from JSON responses
2019-05-07 16:46:02 +00:00
Michael Bideau 01660430af Features
* function 'echofile()' support for posted file content, not just url

Bug fix
    * corrected file content in function 'afile()'
    * corrected payload assignment in function 'get_fwd_files_status()'
    * added 'type_dossier' to job creation in function 'create_dossier()'

Coding style: better indentation
Logging     : 'debug' instead of 'info'
Locale      : identifier in english instead of french
2019-05-06 15:27:25 +00:00