Skip to content

Commit f56d450

Browse files
author
Sylvain MARIE
committed
Major refactoring. Main features:
- **All validation functions are now transformed into failure raisers**, including single callables, and including the code wrapped by `validator` context-manager entry point. `ValidationError.validation_outcome` is therefore renamed `ValidationError.failure` because it is now always guaranteed to be a `Failure`. `Validator` subclasses are now able to specify a `callable_creator`, to override the default `failure_raiser` implementation. Fixes #44 - Validation functions can now either have signature `f(v)`, `f(*args)`, `f(*args, **ctx)` or `f(v, **ctx)`. Fixes #39 - `ValidationFailed` class was merged with `Failure` class for simplification. So there are two main exception classes in valid8 now: `ValidationError` and `ValidationFailure`. When failures are not directly raised by the inner validation callable, `failure_raiser` raises instances of `Invalid` (either `InvalidValue` or `InvalidType` according to the case). Fixes #41 - The string representation of `ValidationError` and `ValidationFailure` was improved. In particular `ValidationError` does not display the name and outcome of the validation function anymore (since it is always a failure), and `ValidationFailure` now has a "compact" string representation option in a new `to_str()` method, used in composition messages to simplify the result. ----Details---- Validation functions: - `_make_validation_func_callable` renamed `make_validation_func_callable` - All validation functions are now transformed into failure raisers by `make_validation_func_callable`, including single callables. - `make_validation_func_callable` can now receive a custom `callable_creator` (default is `failure_raiser`) that is applied to transform each callable to a failure raiser. - validation callables can receive context dictionary if their signature contains a var-keyword arg. For this **all** internal composition and wrapping methods have been modified so as to pass the context along. Also, a new method `make_callable` is called by `failure_raiser` to inspect the provided function signature and wrap it accordingly so as to either pass or not pass the context `**ctx`. A new module `utils_signatures` is used behind the scenes to provide a version of `getfullargspec` capable of supporting partial methods, builtins and bound class and instance methods. `Validator`: - `Validator` is now a class with slots - simplified `Validator._create_validation_error` because we are now sure that the outcome is a `Failure` - `Validator` subclasses are now able to specify a `callable_creator`, to override the default `failure_raiser` - `Validator._create_validation_error` is not anymore responsible for merging the call context and self context. `assert_valid` does. `ValidationError`: - `ValidationError.validation_outcome` renamed `ValidationError.failure` because it is now always guaranteed to be a `Failure` - `ValidationError.create_manually` renamed `ValidationError.create_without_validator` - New method `ValidationError.create_with_dynamic_type`, `add_base_type_dynamically` with a `@lru_cache` to improve performance (added `functools32` dependency for python 2). `ValidationFailed` / `Failure`: - class `ValidationFailed` merged with class `Failure` for simplification, into a `ValidationFailure`. - when failures are not directly raised by the wrapped validation callable, `failure_raiser` raises instances of a subclass of `Invalid`: either `InvalidValue` or `InvalidType` according to the case. Big `ValidationError` and `Failure` string representation simplification - new method `to_str` - removed `display_prefix_for_exc_outcomes` - `compact_mode` argument in `get_details`, to get a - `HelpMsgMixIn` simplification. Improved `HelpMsgFormattingException` to have better error messages in case of formatting issue - `utils_string`: merged util methods `end_with_dot` and `end_with_dot_space`. `validator` context-manager entry point: - replaced `_DummyCallable` with a function. - wrapped code validation errors now raises a failure like in `failure_raiser` - new internal method `_get_wrapped_lines` Misc: - `pop_kwargs` is now in `base` submodule
1 parent 97585d3 commit f56d450

25 files changed

Lines changed: 1393 additions & 763 deletions

setup.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,8 @@
1111
here = path.abspath(path.dirname(__file__))
1212

1313
# *************** Dependencies *********
14-
INSTALL_REQUIRES = ['makefun', 'six', 'future;python_version<"3.3"', 'funcsigs;python_version<"3.3"', 'decopatch'] # 'typing_inspect' is now copied internally so as to be compliant with very old versions of typing module
14+
INSTALL_REQUIRES = ['makefun', 'six', 'future;python_version<"3.3"', 'funcsigs;python_version<"3.3"', 'decopatch',
15+
'functools32;python_version<"3.2"'] # 'typing_inspect' is now copied internally so as to be compliant with very old versions of typing module
1516
DEPENDENCY_LINKS = []
1617
SETUP_REQUIRES = ['pytest-runner', 'setuptools_scm', 'pypandoc', 'pandoc', 'enum34;python_version<"3.4"', 'six']
1718
TESTS_REQUIRE = ['pytest>=4.3.0', 'pytest-logging', 'pytest-cov', 'enforce', 'mini_lambda', 'attrs', 'numpy',

valid8/__init__.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
from valid8.utils_typing import Boolean, is_pep484_nonable
22

3-
from valid8.base import Failure, ValidationFailed, failure_raiser, as_failure_raiser
3+
from valid8.base import ValidationFailure, failure_raiser, as_failure_raiser, Invalid
44
from valid8.composition import CompositionFailure, AtLeastOneFailed, and_, DidNotFail, not_, AllValidatorsFailed, or_, \
55
XorTooManySuccess, xor_, not_all, fail_on_none, skip_on_none
66

@@ -33,7 +33,7 @@
3333
# -- utils_typing
3434
'Boolean', 'is_pep484_nonable',
3535
# -- base
36-
'Failure', 'ValidationFailed', 'failure_raiser', 'as_failure_raiser',
36+
'ValidationFailure', 'Invalid', 'failure_raiser', 'as_failure_raiser',
3737
# -- composition
3838
'CompositionFailure', 'AtLeastOneFailed', 'and_', 'DidNotFail', 'not_', 'AllValidatorsFailed', 'or_',
3939
'XorTooManySuccess', 'xor_', 'not_all', 'fail_on_none', 'skip_on_none',

valid8/base.py

Lines changed: 313 additions & 181 deletions
Large diffs are not rendered by default.

valid8/common_syntax.py

Lines changed: 57 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,9 @@
11
from itertools import chain
22
from sys import version_info
33

4-
from valid8.base import failure_raiser, ValidationFailed, as_function
4+
from makefun import with_signature
5+
6+
from valid8.base import failure_raiser, ValidationFailure, is_mini_lambda, pop_kwargs
57

68
try: # python 3.5+
79
# noinspection PyUnresolvedReferences
@@ -18,14 +20,14 @@
1820
# 2. the syntax to optionally transform them into failure raisers by providing a tuple
1921
ValidationFuncDefinition = Union[ValidationCallableOrLambda,
2022
Tuple[ValidationCallableOrLambda, str],
21-
Tuple[ValidationCallableOrLambda, Type[ValidationFailed]],
22-
Tuple[ValidationCallableOrLambda, str, Type[ValidationFailed]]
23+
Tuple[ValidationCallableOrLambda, Type[ValidationFailure]],
24+
Tuple[ValidationCallableOrLambda, str, Type[ValidationFailure]]
2325
]
2426
"""Defines a checker from a base checker function together with optional error message and failure type
2527
(in which case a failure raiser is created to wrap that function)"""
2628

2729
# 3. the syntax to describe several validation functions at once
28-
VFDefinitionElement = Union[str, Type[ValidationFailed], ValidationCallableOrLambda]
30+
VFDefinitionElement = Union[str, Type[ValidationFailure], ValidationCallableOrLambda]
2931
"""This type represents one of the elements that can define a checker"""
3032

3133
OneOrSeveralVFDefinitions = Union[ValidationFuncDefinition,
@@ -64,32 +66,33 @@ class FunctionDefinitionError(Exception):
6466
'callables, they will be transformed to functions automatically.'
6567

6668

67-
def _make_validation_func_callable(vf_definition # type: ValidationFuncDefinition
68-
):
69+
def make_validation_func_callable(vf_definition, # type: ValidationFuncDefinition
70+
callable_creator=failure_raiser # type: Callable
71+
):
6972
# type: (...) -> ValidationCallable
7073
"""
7174
Creates a validation callable for usage in valid8, from a "validation function" callable optionally completed with
7275
an associated error message and failure type to be used in case validation fails.
7376
74-
If `vf_definition` is a single <validation_func> callable, it is returned directly (no wrapping)
77+
If `vf_definition` is a single <validation_func> callable, a failure raiser is created around it
7578
7679
>>> import sys, pytest
7780
>>> if sys.version_info < (3, 0):
7881
... pytest.skip('doctest skipped in python 2 because exception namespace is different but details matter')
7982
8083
>>> def vf(x): return x + 1 == 0
81-
>>> assert _make_validation_func_callable(vf) is vf
84+
>>> assert make_validation_func_callable(vf) is not vf # TODO better assert here ?
8285
8386
If `vf_definition` is a tuple such as (<validation_func>, <err_msg>), (<validation_func>, <failure_type>),
8487
or (<validation_func>, <err_msg>, <failure_type>), a `failure_raiser` is created.
8588
86-
>>> class MyFailure(ValidationFailed):
89+
>>> class MyFailure(ValidationFailure):
8790
... pass
88-
>>> vf_with_details = _make_validation_func_callable((vf, 'blah', MyFailure))
91+
>>> vf_with_details = make_validation_func_callable((vf, 'blah', MyFailure))
8992
>>> vf_with_details('hello')
9093
Traceback (most recent call last):
9194
...
92-
valid8.common_syntax.MyFailure: blah. Function [vf] raised [TypeError: can...
95+
valid8.common_syntax.MyFailure: blah. Function [vf] raised TypeError: ...
9396
9497
Notes:
9598
@@ -103,6 +106,8 @@ def _make_validation_func_callable(vf_definition # type: ValidationFuncDefiniti
103106
(<validation_func>, <err_msg>), (<validation_func>, <err_type>), or (<validation_func>, <err_msg>, <err_type>)
104107
where <validation_func> is a callable taking a single input and returning `True` or `None` in case of success.
105108
mini-lambda expressions are supported too and automatically converted into callables.
109+
:param callable_creator: method to be called to finally create the callable. Can be used by extensions to specify
110+
a custom way to create callables. Default is `failure_raiser`.
106111
:return: a validation callable that is either directly the provided callable, or a `failure_raiser` wrapping this
107112
callable using the additional details (err_msg, failure_type) provided.
108113
"""
@@ -111,12 +116,12 @@ def _make_validation_func_callable(vf_definition # type: ValidationFuncDefiniti
111116
except (TypeError, FunctionDefinitionError):
112117
# -- single element
113118
# handle the special case of a LambdaExpression: automatically convert to a function
114-
validation_func = as_function(vf_definition)
115-
if not callable(validation_func):
119+
if not is_mini_lambda(vf_definition) and not callable(vf_definition):
116120
raise ValueError('base validation function(s) not compliant with the allowed syntax. Base validation'
117121
' function(s) can be %s Found %s.' % (supported_syntax, vf_definition))
118122
else:
119-
return validation_func
123+
# single element.
124+
return callable_creator(vf_definition)
120125
else:
121126
# -- a tuple
122127
if nb_elts == 1:
@@ -137,7 +142,7 @@ def _make_validation_func_callable(vf_definition # type: ValidationFuncDefiniti
137142
# noinspection PyBroadException
138143
try:
139144
# noinspection PyTypeChecker
140-
if issubclass(failure_type, ValidationFailed):
145+
if issubclass(failure_type, ValidationFailure):
141146
failure_type_ok = True
142147
except: # noqa: E722
143148
pass
@@ -154,30 +159,37 @@ def _make_validation_func_callable(vf_definition # type: ValidationFuncDefiniti
154159
else:
155160
help_msg_ok = True
156161

157-
# handle the special case of a LambdaExpression: automatically convert to a function
158-
# note: it is also done in `failure_raiser` below, but the perf impact should be very small
159-
# (just an instance_of() check)
160-
validation_func = as_function(validation_func)
161-
162162
# check that the definition is valid
163-
if (not failure_type_ok) or (not help_msg_ok) or (not callable(validation_func)):
163+
if (not failure_type_ok) or (not help_msg_ok) or (not is_mini_lambda(validation_func)
164+
and not callable(validation_func)):
164165
raise ValueError('base validation function(s) not compliant with the allowed syntax. Base validation'
165166
' function(s) can be %s Found %s.' % (supported_syntax, vf_definition))
166167

167168
# finally create the failure raising callable
168-
return failure_raiser(validation_func, help_msg=help_msg, failure_type=failure_type)
169+
return callable_creator(validation_func, help_msg=help_msg, failure_type=failure_type)
169170

170171

171-
def _make_validation_func_callables(*vf_definition # type: OneOrSeveralVFDefinitions
172-
):
172+
# Python 3+: load the 'more explicit api'
173+
if use_typing:
174+
new_sig = """(*vf_definition: OneOrSeveralVFDefinitions,
175+
callable_creator: Callable = failure_raiser
176+
)"""
177+
else:
178+
new_sig = None
179+
180+
181+
@with_signature(new_sig)
182+
def make_validation_func_callables(*vf_definition, # type: OneOrSeveralVFDefinitions
183+
**kwargs
184+
):
173185
# type: (...) -> Tuple[ValidationCallable, ...]
174186
"""
175187
Creates one or several validation callables for usage in valid8, from one or several "validation function"
176188
callables, optionally completed with associated error messages and failure types to be used in case validation
177189
fails.
178190
179-
If several `vf_definition` are provided, `_make_validation_func_callable` will be called for each `vf_definition`,
180-
and a tuple containing the results will be returned. See `_make_validation_func_callable` for details on the
191+
If several `vf_definition` are provided, `make_validation_func_callable` will be called for each `vf_definition`,
192+
and a tuple containing the results will be returned. See `make_validation_func_callable` for details on the
181193
supported tuples to use.
182194
183195
>>> import sys, pytest
@@ -186,47 +198,52 @@ def _make_validation_func_callables(*vf_definition # type: OneOrSeveralVFDefini
186198
187199
>>> # two dummy validation callables
188200
>>> def is_big(x): return x > 10
189-
>>> def is_minus_1(x): return x + 1 == 0
201+
>>> def is_minus_1(x, **ctx): return x + 1 == 0
190202
191203
>>> # a custom failure we would like to be raised
192-
>>> class MyFailure(ValidationFailed):
204+
>>> class MyFailure(ValidationFailure):
193205
... pass
194206
195207
>>> # process both vf1 and v2, reusing vf1 'as is' and enriching vf2 with a custom failure type and error message
196-
>>> several_vfs = _make_validation_func_callables([is_big, (is_minus_1, 'not minus 1!', MyFailure)])
208+
>>> several_vfs = make_validation_func_callables([is_big, (is_minus_1, 'not minus 1!', MyFailure)])
197209
>>> assert len(several_vfs) == 2
198-
>>> assert several_vfs[0] is is_big
210+
>>> assert several_vfs[0] is not is_big # TODO better assert here ?
199211
>>> several_vfs[1]('hello')
200212
Traceback (most recent call last):
201213
...
202-
valid8.common_syntax.MyFailure: not minus 1!. Function [is_minus_1] raised [TypeError: can...
214+
valid8.common_syntax.MyFailure: not minus 1!. Function [is_minus_1] raised TypeError: ...
203215
204216
If a single `vf_definition` is provided AND it is a non-tuple iterable (typically a list),
205-
`_make_validation_func_callables(vf_definition)` is equivalent to `_make_validation_func_callables(*vf_definition)`
217+
`make_validation_func_callables(vf_definition)` is equivalent to `make_validation_func_callables(*vf_definition)`
206218
207-
>>> assert _make_validation_func_callables([is_big]) == _make_validation_func_callables(is_big)
219+
# >>> assert make_validation_func_callables([is_big]) == make_validation_func_callables(is_big) TODO re-enable when equality operator works
208220
209221
Finally, if a single `vf_definition` is provided AND it is a dict-like mapping, a special syntax is enabled where
210-
you can put *any* part of the definition in the key and in the value. `_make_validation_func_callable` will still
222+
you can put *any* part of the definition in the key and in the value. `make_validation_func_callable` will still
211223
then be called for each item in the dictionary, and a tuple with the results will be returned.
212224
213225
Examples:
214226
215-
>>> vfs = _make_validation_func_callables({'x should be big': is_big,
227+
>>> vfs = make_validation_func_callables({'x should be big': is_big,
216228
... 'x should be minus 1': (is_minus_1, MyFailure)})
217229
>>> vfs[0](2)
218230
Traceback (most recent call last):
219231
...
220-
valid8.base.ValidationFailed: x should be big. Function [is_big] returned [False] for value 2.
232+
valid8.base.InvalidValue: x should be big. Function [is_big] returned [False] for value 2.
221233
222234
:param vf_definition: the base validation function or list of base validation functions to use. A callable, a
223235
tuple(callable, help_msg_str), a tuple(callable, failure_type), tuple(callable, help_msg_str, failure_type)
224236
or a list of several such elements.
225237
Tuples indicate an implicit `failure_raiser`.
226238
[mini_lambda](https://smarie.github.io/python-mini-lambda/) expressions can be used instead
227239
of callables, they will be transformed to functions automatically.
240+
:param callable_creator: method to be called to finally create the callable. Can be used by extensions to specify
241+
a custom way to create callables. Default is `failure_raiser`.
228242
:return: a tuple of callables
229243
"""
244+
# python <3.5 compliance: pop the kwargs following the varargs
245+
callable_creator = pop_kwargs(kwargs, [('callable_creator', failure_raiser)], allow_others=False)
246+
230247
# handle the case where vf_definition is not yet a list or is empty or none
231248
if len(vf_definition) == 0:
232249
raise ValueError('mandatory vf_definition is None')
@@ -245,10 +262,10 @@ def _make_validation_func_callables(*vf_definition # type: OneOrSeveralVFDefini
245262
v_iter = iter(vf_definition)
246263
except (TypeError, FunctionDefinitionError):
247264
# single validator: create a tuple manually
248-
all_validators = (_make_validation_func_callable(vf_definition),)
265+
all_validators = (make_validation_func_callable(vf_definition, callable_creator=callable_creator),)
249266
else:
250267
# iterable
251-
all_validators = tuple(_make_validation_func_callable(v) for v in v_iter)
268+
all_validators = tuple(make_validation_func_callable(v, callable_creator=callable_creator) for v in v_iter)
252269
else:
253270
# mapping: be 'smart'
254271
def _mapping_entry_to_vf(k, v):
@@ -270,14 +287,14 @@ def _mapping_entry_to_vf(k, v):
270287
err_msg = _elt
271288
else:
272289
try:
273-
if issubclass(_elt, Exception): # not Failure so that we reuse the check that is made below
290+
if issubclass(_elt, Exception): # broad: Exception, check for ValidationFailure subclass is made below
274291
err_type = _elt
275292
else:
276293
callabl = _elt
277294
except TypeError:
278295
callabl = _elt
279296

280-
return _make_validation_func_callable((callabl, err_msg, err_type))
297+
return make_validation_func_callable((callabl, err_msg, err_type), callable_creator=callable_creator)
281298

282299
all_validators = tuple(_mapping_entry_to_vf(k, v) for k, v in v_items)
283300

0 commit comments

Comments
 (0)