21 Commits
0.6.3 ... 0.9.0

Author SHA1 Message Date
4c385667e8 Fix settings being overwritten when using the same dest attribute 2023-12-16 16:51:00 -08:00
9ffefb3e21 Fix readme example 2023-11-19 00:20:43 -08:00
23d0139144 Map argparse --help output to python versions 2023-11-19 00:14:06 -08:00
0c49b9309d Persist unknown groups 2023-11-18 23:58:18 -08:00
ccacca1b32 Improve generated namespace
Improve formatting of namespace
Allow _guess_type to return any string
Add tests
2023-11-18 23:31:24 -08:00
2c79e62765 Fix exported names 2023-11-18 12:43:53 -08:00
27ca830b60 [pre-commit.ci] pre-commit autoupdate
updates:
- [github.com/pre-commit/pre-commit-hooks: v4.4.0 → v4.5.0](https://github.com/pre-commit/pre-commit-hooks/compare/v4.4.0...v4.5.0)
- [github.com/asottile/setup-cfg-fmt: v2.4.0 → v2.5.0](https://github.com/asottile/setup-cfg-fmt/compare/v2.4.0...v2.5.0)
- [github.com/asottile/reorder-python-imports: v3.10.0 → v3.12.0](https://github.com/asottile/reorder-python-imports/compare/v3.10.0...v3.12.0)
- [github.com/asottile/pyupgrade: v3.10.1 → v3.15.0](https://github.com/asottile/pyupgrade/compare/v3.10.1...v3.15.0)
- [github.com/pre-commit/mirrors-mypy: v1.5.1 → v1.7.0](https://github.com/pre-commit/mirrors-mypy/compare/v1.5.1...v1.7.0)
2023-11-13 17:18:39 +00:00
101eef56ca Sanitize group names 2023-09-04 18:32:14 -05:00
2f000e12f3 [pre-commit.ci] pre-commit autoupdate
updates:
- [github.com/asottile/add-trailing-comma: v3.0.1 → v3.1.0](https://github.com/asottile/add-trailing-comma/compare/v3.0.1...v3.1.0)
- https://github.com/pre-commit/mirrors-autopep8https://github.com/hhatto/autopep8
- [github.com/hhatto/autopep8: v2.0.2 → v2.0.4](https://github.com/hhatto/autopep8/compare/v2.0.2...v2.0.4)
- [github.com/pre-commit/mirrors-mypy: v1.5.0 → v1.5.1](https://github.com/pre-commit/mirrors-mypy/compare/v1.5.0...v1.5.1)
2023-09-04 19:22:46 +00:00
0301e1698c Merge pull request #5 from lordwelch/pre-commit-ci-update-config
[pre-commit.ci] pre-commit autoupdate
2023-08-15 07:43:05 -07:00
6af1e3c562 [pre-commit.ci] pre-commit autoupdate
updates:
- [github.com/asottile/setup-cfg-fmt: v2.3.0 → v2.4.0](https://github.com/asottile/setup-cfg-fmt/compare/v2.3.0...v2.4.0)
- [github.com/asottile/add-trailing-comma: v2.5.1 → v3.0.1](https://github.com/asottile/add-trailing-comma/compare/v2.5.1...v3.0.1)
- [github.com/asottile/pyupgrade: v3.7.0 → v3.10.1](https://github.com/asottile/pyupgrade/compare/v3.7.0...v3.10.1)
- [github.com/PyCQA/flake8: 6.0.0 → 6.1.0](https://github.com/PyCQA/flake8/compare/6.0.0...6.1.0)
- [github.com/pre-commit/mirrors-mypy: v1.3.0 → v1.5.0](https://github.com/pre-commit/mirrors-mypy/compare/v1.3.0...v1.5.0)
2023-08-14 19:13:59 +00:00
fe3bce42fd [pre-commit.ci] pre-commit autoupdate
updates:
- [github.com/asottile/reorder-python-imports: v3.9.0 → v3.10.0](https://github.com/asottile/reorder-python-imports/compare/v3.9.0...v3.10.0)
- [github.com/asottile/add-trailing-comma: v2.4.0 → v2.5.1](https://github.com/asottile/add-trailing-comma/compare/v2.4.0...v2.5.1)
- [github.com/asottile/dead: v1.5.1 → v1.5.2](https://github.com/asottile/dead/compare/v1.5.1...v1.5.2)
- [github.com/asottile/pyupgrade: v3.4.0 → v3.7.0](https://github.com/asottile/pyupgrade/compare/v3.4.0...v3.7.0)
2023-06-19 19:05:50 +00:00
4c41e6f588 Use a custom class for typing
Add support for nargs to default to list[str]
2023-06-09 15:41:18 -07:00
b599097cc1 Adds a function to generate a class for typing a namespace 2023-06-08 22:54:36 -07:00
8af75d3962 Update docstrings and ensure parameters are consistent 2023-06-08 21:44:07 -07:00
e57ee25a60 Make get_namespace consistent with normalize_config 2023-06-08 21:44:07 -07:00
14357c14de [pre-commit.ci] pre-commit autoupdate
updates:
- [github.com/asottile/setup-cfg-fmt: v2.2.0 → v2.3.0](https://github.com/asottile/setup-cfg-fmt/compare/v2.2.0...v2.3.0)
- https://github.com/asottile/reorder_python_importshttps://github.com/asottile/reorder-python-imports
- [github.com/asottile/dead: v1.5.0 → v1.5.1](https://github.com/asottile/dead/compare/v1.5.0...v1.5.1)
- [github.com/asottile/pyupgrade: v3.3.2 → v3.4.0](https://github.com/asottile/pyupgrade/compare/v3.3.2...v3.4.0)
- [github.com/pre-commit/mirrors-mypy: v1.2.0 → v1.3.0](https://github.com/pre-commit/mirrors-mypy/compare/v1.2.0...v1.3.0)
2023-06-05 18:41:50 +00:00
3ed9571156 Merge pull request #2 from lordwelch/pre-commit-ci-update-config
[pre-commit.ci] pre-commit autoupdate
2023-05-06 09:32:31 -07:00
c76ded2c4f [pre-commit.ci] pre-commit autoupdate
updates:
- [github.com/asottile/pyupgrade: v3.3.1 → v3.3.2](https://github.com/asottile/pyupgrade/compare/v3.3.1...v3.3.2)
2023-05-01 18:48:23 +00:00
78ac274e31 Include pyproject.toml 2023-04-26 15:32:06 -07:00
05c2196fcf Update minimum requirements for typing-extensions 2023-04-26 15:04:57 -07:00
8 changed files with 917 additions and 271 deletions

View File

@ -33,7 +33,7 @@ jobs:
- name: Build and install wheel
run: |
python -m build
tox run -m build
python -m pip install dist/*.whl
- name: tox

View File

@ -1,6 +1,6 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
rev: v4.5.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
@ -10,38 +10,38 @@ repos:
- id: name-tests-test
- id: requirements-txt-fixer
- repo: https://github.com/asottile/setup-cfg-fmt
rev: v2.2.0
rev: v2.5.0
hooks:
- id: setup-cfg-fmt
- repo: https://github.com/asottile/reorder_python_imports
rev: v3.9.0
- repo: https://github.com/asottile/reorder-python-imports
rev: v3.12.0
hooks:
- id: reorder-python-imports
args: [--py38-plus, --add-import, 'from __future__ import annotations']
- repo: https://github.com/asottile/add-trailing-comma
rev: v2.4.0
rev: v3.1.0
hooks:
- id: add-trailing-comma
args: [--py36-plus]
- repo: https://github.com/asottile/dead
rev: v1.5.0
rev: v1.5.2
hooks:
- id: dead
- repo: https://github.com/asottile/pyupgrade
rev: v3.3.1
rev: v3.15.0
hooks:
- id: pyupgrade
args: [--py38-plus]
- repo: https://github.com/pre-commit/mirrors-autopep8
rev: v2.0.2
- repo: https://github.com/hhatto/autopep8
rev: v2.0.4
hooks:
- id: autopep8
- repo: https://github.com/PyCQA/flake8
rev: 6.0.0
rev: 6.1.0
hooks:
- id: flake8
additional_dependencies: [flake8-encodings, flake8-warnings, flake8-builtins, flake8-length, flake8-print]
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.2.0
rev: v1.7.0
hooks:
- id: mypy

View File

@ -29,14 +29,14 @@ $ python -m settngs
Hello lordwelch
$ python -m settngs -v
Hello lordwelch
merged_namespace.values.example_verbose=True
merged_namespace.values.Example_Group__verbose=True
$ python -m settngs -v -s
Hello lordwelch
Successfully saved settings to settings.json
merged_namespace.values.example_verbose=True
merged_namespace.values.Example_Group__verbose=True
$ python -m settngs
Hello lordwelch
merged_namespace.values.example_verbose=True
merged_namespace.values.Example_Group__verbose=True
$ cat >settings.json << EOF
{
"example": {
@ -64,7 +64,7 @@ Hello world
settngs.json at the end:
```json
{
"example": {
"Example Group": {
"hello": "world",
"verbose": false
},

6
pyproject.toml Normal file
View File

@ -0,0 +1,6 @@
[build-system]
requires = ["setuptools>=42", "wheel", "setuptools_scm[toml]>=3.4"]
build-backend = "setuptools.build_meta"
[tool.setuptools_scm]
local_scheme = "no-local-version"

View File

@ -1,11 +1,13 @@
from __future__ import annotations
import argparse
import copy
import json
import logging
import pathlib
import re
import sys
import typing
from argparse import Namespace
from collections import defaultdict
from collections.abc import Sequence
@ -14,10 +16,10 @@ from typing import Callable
from typing import Dict
from typing import Generic
from typing import NoReturn
from typing import overload
from typing import TYPE_CHECKING
from typing import TypeVar
from typing import Union
logger = logging.getLogger(__name__)
if sys.version_info < (3, 11): # pragma: no cover
@ -27,6 +29,9 @@ else: # pragma: no cover
if sys.version_info < (3, 9): # pragma: no cover
from typing import List
from typing import _GenericAlias as types_GenericAlias
def removeprefix(self: str, prefix: str, /) -> str:
if self.startswith(prefix):
return self[len(prefix):]
@ -73,6 +78,8 @@ if sys.version_info < (3, 9): # pragma: no cover
if option_string in self.option_strings:
setattr(namespace, self.dest, not option_string.startswith('--no-'))
else: # pragma: no cover
List = list
from types import GenericAlias as types_GenericAlias
from argparse import BooleanOptionalAction
removeprefix = str.removeprefix
@ -82,10 +89,10 @@ class Setting:
self,
# From argparse
*names: str,
action: type[argparse.Action] | None = None,
action: type[argparse.Action] | str | None = None,
nargs: str | int | None = None,
const: str | None = None,
default: str | None = None,
const: Any | None = None,
default: Any | None = None,
type: Callable[..., Any] | None = None, # noqa: A002
choices: Sequence[Any] | None = None,
required: bool | None = None,
@ -100,6 +107,9 @@ class Setting:
exclusive: bool = False,
):
"""
Attributes:
setting_name: This is the name used to retrieve this Setting object from a `Config` Definitions dictionary.
This only differs from dest when a custom dest is given
Args:
*names: Passed directly to argparse
@ -111,8 +121,9 @@ class Setting:
choices: Passed directly to argparse
required: Passed directly to argparse
help: Passed directly to argparse
metavar: Passed directly to argparse, defaults to `dest` uppercased
dest: This is the name used to retrieve the value from a `Config` object as a dictionary
metavar: Passed directly to argparse, defaults to `dest` upper-cased
dest: This is the name used to retrieve the value from a `Config` object as a dictionary.
Default to `setting_name`.
display_name: This is not used by settngs. This is a human-readable name to be used when generating a GUI.
Defaults to `dest`.
cmdline: If this setting can be set via the commandline
@ -126,7 +137,7 @@ class Setting:
raise ValueError('names must be specified')
# We prefix the destination name used by argparse so that there are no conflicts
# Argument names will still cause an exception if there is a conflict e.g. if '-f' is defined twice
self.internal_name, dest, flag = self.get_dest(group, names, dest)
self.internal_name, setting_name, dest, self.flag = self.get_dest(group, names, dest)
args: Sequence[str] = names
# We then also set the metavar so that '--config' in the group runtime shows as 'CONFIG' instead of 'RUNTIME_CONFIG'
@ -134,10 +145,9 @@ class Setting:
metavar = dest.upper()
# If we are not a flag, no '--' or '-' in front
# we prefix the first name with the group as argparse sets dest to args[0]
# I believe internal name may be able to be used here
if not flag:
args = tuple((f'{group}_{names[0]}'.lstrip('_'), *names[1:]))
# we use internal_name as argparse sets dest to args[0]
if not self.flag:
args = tuple((self.internal_name, *names[1:]))
self.action = action
self.nargs = nargs
@ -149,6 +159,7 @@ class Setting:
self.help = help
self.metavar = metavar
self.dest = dest
self.setting_name = setting_name
self.cmdline = cmdline
self.file = file
self.argparse_args = args
@ -166,7 +177,7 @@ class Setting:
'required': required,
'help': help,
'metavar': metavar,
'dest': self.internal_name if flag else None,
'dest': self.internal_name if self.flag else None,
}
def __str__(self) -> str: # pragma: no cover
@ -180,27 +191,72 @@ class Setting:
return NotImplemented
return self.__dict__ == other.__dict__
def get_dest(self, prefix: str, names: Sequence[str], dest: str | None) -> tuple[str, str, bool]:
dest_name = None
def _guess_type(self) -> type | str | None:
if self.type is None and self.action is None:
if self.cmdline:
if self.nargs in ('+', '*') or isinstance(self.nargs, int) and self.nargs > 1:
return List[str]
return str
else:
if not self.cmdline and self.default is not None:
return type(self.default)
return 'Any'
if isinstance(self.type, type):
return self.type
if self.type is not None:
type_hints = typing.get_type_hints(self.type)
if 'return' in type_hints:
t: type | str = type_hints['return']
return t
if self.default is not None:
return type(self.default)
return 'Any'
if self.action in ('store_true', 'store_false', BooleanOptionalAction):
return bool
if self.action in ('store_const',):
return type(self.const)
if self.action in ('count',):
return int
if self.action in ('append', 'extend'):
return List[str]
if self.action in ('append_const',):
return list # list[type(self.const)]
if self.action in ('help', 'version'):
return None
return 'Any'
def get_dest(self, prefix: str, names: Sequence[str], dest: str | None) -> tuple[str, str, str, bool]:
setting_name = None
flag = False
prefix = sanitize_name(prefix)
for n in names:
if n.startswith('--'):
flag = True
dest_name = sanitize_name(n)
setting_name = sanitize_name(n)
break
if n.startswith('-'):
flag = True
if dest_name is None:
dest_name = names[0]
if setting_name is None:
setting_name = names[0]
if dest:
dest_name = dest
else:
dest_name = setting_name
if not dest_name.isidentifier():
raise Exception(f'Cannot use {dest_name} in a namespace')
internal_name = f'{prefix}_{dest_name}'.lstrip('_')
return internal_name, dest_name, flag
internal_name = f'{prefix}__{dest_name}'.lstrip('_')
return internal_name, setting_name, dest_name, flag
def filter_argparse_kwargs(self) -> dict[str, Any]:
return {k: v for k, v in self.argparse_kwargs.items() if v is not None}
@ -209,6 +265,11 @@ class Setting:
return self.argparse_args, self.filter_argparse_kwargs()
class TypedNS:
def __init__(self) -> None:
raise TypeError('TypedNS cannot be instantiated')
class Group(NamedTuple):
persistent: bool
v: dict[str, Setting]
@ -217,7 +278,7 @@ class Group(NamedTuple):
Values = Dict[str, Dict[str, Any]]
Definitions = Dict[str, Group]
T = TypeVar('T', Values, Namespace)
T = TypeVar('T', bound=Union[Values, Namespace, TypedNS])
class Config(NamedTuple, Generic[T]):
@ -227,16 +288,71 @@ class Config(NamedTuple, Generic[T]):
if TYPE_CHECKING:
ArgParser = Union[argparse._MutuallyExclusiveGroup, argparse._ArgumentGroup, argparse.ArgumentParser]
ns = Namespace | Config[T] | None
ns = Namespace | TypedNS | Config[T] | None
def generate_ns(definitions: Definitions) -> str:
initial_imports = ['from __future__ import annotations', '', 'import settngs', '']
imports: Sequence[str] | set[str]
imports = set()
attributes = []
for group in definitions.values():
for setting in group.v.values():
t = setting._guess_type()
if t is None:
continue
# Default to any
type_name = 'Any'
# Take a string as is
if isinstance(t, str):
type_name = t
# Handle generic aliases eg dict[str, str] instead of dict
elif isinstance(t, types_GenericAlias):
type_name = str(t)
# Handle standard type objects
elif isinstance(t, type):
type_name = t.__name__
# Builtin types don't need an import
if t.__module__ != 'builtins':
imports.add(f'import {t.__module__}')
# Use the full imported name
type_name = t.__module__ + '.' + type_name
# Expand Any to typing.Any
if type_name == 'Any':
type_name = 'typing.Any'
attributes.append(f' {setting.internal_name}: {type_name}')
# Add a blank line between groups
if attributes and attributes[-1] != '':
attributes.append('')
ns = 'class settngs_namespace(settngs.TypedNS):\n'
# Add a '...' expression if there are no attributes
if not attributes or all(x == '' for x in attributes):
ns += ' ...\n'
attributes = ['']
# Add the tying import before extra imports
if 'typing.' in '\n'.join(attributes):
initial_imports.append('import typing')
# Remove the possible duplicate typing import
imports = sorted(list(imports - {'import typing'}))
# Merge the imports the ns class definition and the attributes
return '\n'.join(initial_imports + imports) + '\n\n\n' + ns + '\n'.join(attributes)
def sanitize_name(name: str) -> str:
return re.sub('[' + re.escape(' -_,.!@#$%^&*(){}[]\',."<>;:') + ']+', '_', name).strip('_')
def get_option(options: Values | Namespace, setting: Setting) -> tuple[Any, bool]:
def get_option(options: Values | Namespace | TypedNS, setting: Setting) -> tuple[Any, bool]:
"""
Helper function to retrieve the value for a setting and if the value is the default value
Helper function to retrieve the value for a setting and if the current value is the default value
Args:
options: Dictionary or namespace of options
@ -249,35 +365,58 @@ def get_option(options: Values | Namespace, setting: Setting) -> tuple[Any, bool
return value, value == setting.default
def get_options(options: Config[T], group: str) -> dict[str, Any]:
def get_options(config: Config[T], group: str) -> dict[str, Any]:
"""
Helper function to retrieve all of the values for a group. Only to be used on persistent groups.
Args:
options: Dictionary or namespace of options
config: Dictionary or namespace of options
group: The name of the group to retrieve
"""
if isinstance(options[0], dict):
values = options[0].get(group, {}).copy()
if isinstance(config[0], dict):
values: dict[str, Any] = config[0].get(group, {}).copy()
else:
internal_names = {x.internal_name: x for x in options[1][group].v.values()}
internal_names = {x.internal_name: x for x in config[1][group].v.values()}
values = {}
v = vars(options[0])
v = vars(config[0])
for name, value in v.items():
if name.startswith(f'{group}_'):
if name in internal_names:
values[internal_names[name].dest] = value
else:
values[removeprefix(name, f'{group}_')] = value
values[removeprefix(name, f'{group}').lstrip('_')] = value
return values
def get_groups(values: Values | Namespace | TypedNS) -> list[str]:
if isinstance(values, dict):
return [x[0] for x in values.items() if isinstance(x[1], dict)]
if isinstance(values, Namespace):
groups = set()
for name in values.__dict__:
if '__' in name:
group, _, _ = name.partition('__')
groups.add(group.replace('_', ' '))
else:
groups.add('')
return list(groups)
return []
def _get_internal_definitions(config: Config[T], persistent: bool) -> Definitions:
definitions = copy.deepcopy(dict(config.definitions))
if persistent:
for group_name in get_groups(config.values):
if group_name not in definitions:
definitions[group_name] = Group(True, {})
return defaultdict(lambda: Group(False, {}), definitions)
def normalize_config(
config: Config[T],
file: bool = False,
cmdline: bool = False,
defaults: bool = True,
default: bool = True,
persistent: bool = True,
) -> Config[Values]:
"""
@ -286,41 +425,49 @@ def normalize_config(
Values are assigned so if the value is a dictionary mutating it will mutate the original.
Args:
raw_options: The dict or Namespace to normalize options from
definitions: The definition of the options
config: The Config object to normalize options from
file: Include file options
cmdline: Include cmdline options
defaults: Include default values in the returned dict
persistent: Include unknown keys in persistent groups
default: Include default values in the returned Config object
persistent: Include unknown keys in persistent groups and unknown groups
"""
if not file and not cmdline:
raise ValueError('Invalid parameters: you must set either file or cmdline to True')
normalized: Values = {}
options, definitions = config
options = config.values
definitions = _get_internal_definitions(config=config, persistent=persistent)
for group_name, group in definitions.items():
group_options = {}
if group.persistent and persistent:
group_options = get_options(config, group_name)
group_options = get_options(Config(options, definitions), group_name)
for setting_name, setting in group.v.items():
if (setting.cmdline and cmdline) or (setting.file and file):
# Ensures the option exists with the default if not already set
value, default = get_option(options, setting)
if not default or (default and defaults):
value, is_default = get_option(options, setting)
if not is_default or default:
# User has set a custom value or has requested the default value
group_options[setting_name] = value
elif setting_name in group_options:
# defaults have been requested to be removed
# default values have been requested to be removed
del group_options[setting_name]
elif setting_name in group_options:
# Setting type (file or cmdline) has not been requested and should be removed for persistent groups
del group_options[setting_name]
normalized[group_name] = group_options
return Config(normalized, definitions)
return Config(normalized, config.definitions)
def parse_file(definitions: Definitions, filename: pathlib.Path) -> tuple[Config[Values], bool]:
"""
Helper function to read options from a json dictionary from a file
Helper function to read options from a json dictionary from a file.
This is purely a convenience function.
If _anything_ more advanced is desired this should be handled by the application.
Args:
definitions: A set of setting definitions. See `Config.definitions` and `Manager.definitions`
filename: A pathlib.Path object to read a json dictionary from
"""
options: Values = {}
@ -331,82 +478,91 @@ def parse_file(definitions: Definitions, filename: pathlib.Path) -> tuple[Config
opts = json.load(file)
if isinstance(opts, dict):
options = opts
except Exception:
else: # pragma: no cover
raise Exception('Loaded file is not a JSON Dictionary')
except Exception: # pragma: no cover
logger.exception('Failed to load config file: %s', filename)
success = False
else:
logger.info('No config file found')
success = True
return (normalize_config(Config(options, definitions), file=True), success)
return normalize_config(Config(options, definitions), file=True), success
def clean_config(
config: Config[T], file: bool = False, cmdline: bool = False,
config: Config[T], file: bool = False, cmdline: bool = False, default: bool = True, persistent: bool = True,
) -> Values:
"""
Normalizes options and then cleans up empty groups
Normalizes options and then cleans up empty groups. The returned value is probably JSON serializable.
Args:
options:
file:
cmdline:
Returns:
config: The Config object to normalize options from
file: Include file options
cmdline: Include cmdline options
default: Include default values in the returned Config object
persistent: Include unknown keys in persistent groups and unknown groups
"""
clean_options, definitions = normalize_config(config, file=file, cmdline=cmdline)
for group in list(clean_options.keys()):
if not clean_options[group]:
del clean_options[group]
return clean_options
cleaned, _ = normalize_config(config, file=file, cmdline=cmdline, default=default, persistent=persistent)
for group in list(cleaned.keys()):
if not cleaned[group]:
del cleaned[group]
return cleaned
def defaults(definitions: Definitions) -> Config[Values]:
return normalize_config(Config(Namespace(), definitions), file=True, cmdline=True)
def get_namespace(config: Config[T], defaults: bool = True, persistent: bool = True) -> Config[Namespace]:
def get_namespace(
config: Config[T], file: bool = False, cmdline: bool = False, default: bool = True, persistent: bool = True,
) -> Config[Namespace]:
"""
Returns an Namespace object with options in the form "{group_name}_{setting_name}"
`options` should already be normalized.
Throws an exception if the internal_name is duplicated
Returns a Namespace object with options in the form "{group_name}_{setting_name}"
`config` should already be normalized or be a `Config[Namespace]`.
Args:
options: Normalized options to turn into a Namespace
defaults: Include default values in the returned dict
persistent: Include unknown keys in persistent groups
config: The Config object to turn into a namespace
file: Include file options
cmdline: Include cmdline options
default: Include default values in the returned Config object
persistent: Include unknown keys in persistent groups and unknown groups
"""
if not file and not cmdline:
raise ValueError('Invalid parameters: you must set either file or cmdline to True')
if isinstance(config.values, Namespace):
options, definitions = normalize_config(config, True, True, defaults=defaults, persistent=persistent)
options: Values
definitions = _get_internal_definitions(config=config, persistent=persistent)
if isinstance(config.values, dict):
options = config.values
else:
options, definitions = config
cfg = normalize_config(config, file=file, cmdline=cmdline, default=default, persistent=persistent)
options = cfg.values
namespace = Namespace()
for group_name, group in definitions.items():
group_options = get_options(Config(options, definitions), group_name)
if group.persistent and persistent:
group_options = get_options(config, group_name)
for name, value in group_options.items():
if name in group.v:
internal_name, default = group.v[name].internal_name, group.v[name].default == value
setting_file, setting_cmdline = group.v[name].file, group.v[name].cmdline
value, is_default = get_option(options, group.v[name])
internal_name = group.v[name].internal_name
else:
internal_name, default = f'{group_name}_' + sanitize_name(name), None
setting_file = setting_cmdline = True
internal_name, is_default = f'{group_name}__' + sanitize_name(name), None
if hasattr(namespace, internal_name):
raise Exception(f'Duplicate internal name: {internal_name}')
if not default or default and defaults:
if ((setting_cmdline and cmdline) or (setting_file and file)) and (not is_default or default):
setattr(namespace, internal_name, value)
else:
for setting_name, setting in group.v.items():
if hasattr(namespace, setting.internal_name):
raise Exception(f'Duplicate internal name: {setting.internal_name}')
value, default = get_option(options, setting)
for setting in group.v.values():
if (setting.cmdline and cmdline) or (setting.file and file):
value, is_default = get_option(options, setting)
if not default or default and defaults:
if not is_default or default:
# User has set a custom value or has requested the default value
setattr(namespace, setting.internal_name, value)
return Config(namespace, definitions)
return Config(namespace, config.definitions)
def save_file(
@ -414,16 +570,19 @@ def save_file(
) -> bool:
"""
Helper function to save options from a json dictionary to a file
This is purely a convenience function.
If _anything_ more advanced is desired this should be handled by the application.
Args:
options: The options to save to a json dictionary
config: The options to save to a json dictionary
filename: A pathlib.Path object to save the json dictionary to
"""
file_options = clean_config(config, file=True)
try:
if not filename.exists():
filename.parent.mkdir(exist_ok=True, parents=True)
filename.touch()
try:
json_str = json.dumps(file_options, indent=2)
filename.write_text(json_str + '\n', encoding='utf-8')
except Exception:
@ -438,8 +597,8 @@ def create_argparser(definitions: Definitions, description: str, epilog: str) ->
argparser = argparse.ArgumentParser(
description=description, epilog=epilog, formatter_class=argparse.RawTextHelpFormatter,
)
for group_name, group in definitions.items():
for setting_name, setting in group.v.items():
for group in definitions.values():
for setting in group.v.values():
if setting.cmdline:
argparse_args, argparse_kwargs = setting.to_argparse()
current_group: ArgParser = argparser
@ -452,8 +611,11 @@ def create_argparser(definitions: Definitions, description: str, epilog: str) ->
else:
groups[setting.group] = argparser.add_argument_group(setting.group)
# hard coded exception for files
if not (setting.group == 'runtime' and setting.nargs == '*'):
# Hard coded exception for positional arguments
# Ensures that the option shows at the top of the help output
if 'runtime' in setting.group.casefold() and setting.nargs == '*' and not setting.flag:
current_group = argparser
else:
current_group = groups[setting.group]
current_group.add_argument(*argparse_args, **argparse_kwargs)
return argparser
@ -471,17 +633,21 @@ def parse_cmdline(
`args` and `namespace` are passed to `argparse.ArgumentParser.parse_args`
Args:
args: Passed to argparse.ArgumentParser.parse
namespace: Passed to argparse.ArgumentParser.parse
definitions: A set of setting definitions. See `Config.definitions` and `Manager.definitions`
description: Passed to argparse.ArgumentParser
epilog: Passed to argparse.ArgumentParser
args: Passed to argparse.ArgumentParser.parse_args
config: The Config or Namespace object to use as a Namespace passed to argparse.ArgumentParser.parse_args
"""
namespace = None
namespace: Namespace | TypedNS | None = None
if isinstance(config, Config):
if isinstance(config.values, Namespace):
namespace = config.values
else:
namespace = get_namespace(config, defaults=False)[0]
namespace = get_namespace(config, file=True, cmdline=True, default=False)[0]
else:
namespace = config
argparser = create_argparser(definitions, description, epilog)
ns = argparser.parse_args(args, namespace=namespace)
@ -495,13 +661,25 @@ def parse_config(
config_path: pathlib.Path,
args: list[str] | None = None,
) -> tuple[Config[Values], bool]:
"""
Convenience function to parse options from a json file and passes the resulting Config object to parse_cmdline.
This is purely a convenience function.
If _anything_ more advanced is desired this should be handled by the application.
Args:
definitions: A set of setting definitions. See `Config.definitions` and `Manager.definitions`
description: Passed to argparse.ArgumentParser
epilog: Passed to argparse.ArgumentParser
config_path: A `pathlib.Path` object
args: Passed to argparse.ArgumentParser.parse_args
"""
file_options, success = parse_file(definitions, config_path)
cmdline_options = parse_cmdline(
definitions, description, epilog, args, get_namespace(file_options, defaults=False),
definitions, description, epilog, args, file_options,
)
final_options = normalize_config(cmdline_options, file=True, cmdline=True)
return (final_options, success)
return final_options, success
class Manager:
@ -513,21 +691,31 @@ class Manager:
self.description = description
self.epilog = epilog
self.definitions: Definitions
if isinstance(definitions, Config):
self.definitions = definitions.definitions
self.definitions = defaultdict(lambda: Group(False, {}), dict(definitions.definitions) or {})
else:
self.definitions = defaultdict(lambda: Group(False, {}), definitions or {})
self.definitions = defaultdict(lambda: Group(False, {}), dict(definitions or {}))
self.exclusive_group = False
self.current_group_name = ''
def _get_config(self, c: T | Config[T]) -> Config[T]:
if not isinstance(c, Config):
return Config(c, self.definitions)
return c
def generate_ns(self) -> str:
return generate_ns(self.definitions)
def create_argparser(self) -> None:
self.argparser = create_argparser(self.definitions, self.description, self.epilog)
def add_setting(self, *args: Any, **kwargs: Any) -> None:
"""Takes passes all arguments through to `Setting`, `group` and `exclusive` are already set"""
"""Passes all arguments through to `Setting`, `group` and `exclusive` are already set"""
setting = Setting(*args, **kwargs, group=self.current_group_name, exclusive=self.exclusive_group)
self.definitions[self.current_group_name].v[setting.dest] = setting
self.definitions[self.current_group_name].v[setting.setting_name] = setting
def add_group(self, name: str, group: Callable[[Manager], None], exclusive_group: bool = False) -> None:
"""
@ -538,6 +726,7 @@ class Manager:
group: A function that registers individual options using :meth:`add_setting`
exclusive_group: If this group is an argparse exclusive group
"""
if self.current_group_name != '':
raise ValueError('Sub groups are not allowed')
self.current_group_name = name
@ -556,13 +745,14 @@ class Manager:
group: A function that registers individual options using :meth:`add_setting`
exclusive_group: If this group is an argparse exclusive group
"""
if self.current_group_name != '':
raise ValueError('Sub groups are not allowed')
self.current_group_name = name
self.exclusive_group = exclusive_group
if self.current_group_name in self.definitions:
if not self.definitions[self.current_group_name].persistent:
raise ValueError('Group already existis and is not persistent')
raise ValueError('Group already exists and is not persistent')
else:
self.definitions[self.current_group_name] = Group(True, {})
group(self)
@ -578,63 +768,146 @@ class Manager:
return defaults(self.definitions)
def clean_config(
self, options: T | Config[T], file: bool = False, cmdline: bool = False,
self, config: T | Config[T], file: bool = False, cmdline: bool = False,
) -> Values:
if isinstance(options, Config):
config = options
else:
config = Config(options, self.definitions)
return clean_config(config, file=file, cmdline=cmdline)
"""
Normalizes options and then cleans up empty groups. The returned value is probably JSON serializable.
Args:
config: The Config object to normalize options from
file: Include file options
cmdline: Include cmdline options
"""
return clean_config(self._get_config(config), file=file, cmdline=cmdline)
def normalize_config(
self,
options: T | Config[T],
config: T | Config[T],
file: bool = False,
cmdline: bool = False,
defaults: bool = True,
default: bool = True,
persistent: bool = True,
) -> Config[Values]:
if isinstance(options, Config):
config = options
else:
config = Config(options, self.definitions)
"""
Creates an `OptionValues` dictionary with setting definitions taken from `self.definitions`
and values taken from `raw_options` and `raw_options_2' if defined.
Values are assigned so if the value is a dictionary mutating it will mutate the original.
Args:
config: The Config object to normalize options from
file: Include file options
cmdline: Include cmdline options
default: Include default values in the returned Config object
persistent: Include unknown keys in persistent groups and unknown groups
"""
return normalize_config(
config=config,
config=self._get_config(config),
file=file,
cmdline=cmdline,
defaults=defaults,
default=default,
persistent=persistent,
)
@overload
def get_namespace(self, options: Values, defaults: bool = True) -> Namespace:
...
def get_namespace(
self,
config: T | Config[T],
file: bool = False,
cmdline: bool = False,
default: bool = True,
persistent: bool = True,
) -> Config[Namespace]:
"""
Returns a Namespace object with options in the form "{group_name}_{setting_name}"
`options` should already be normalized or be a `Config[Namespace]`.
Throws an exception if the internal_name is duplicated
@overload
def get_namespace(self, options: Config[Values], defaults: bool = True) -> Config[Namespace]:
...
Args:
config: The Config object to turn into a namespace
file: Include file options
cmdline: Include cmdline options
default: Include default values in the returned Config object
persistent: Include unknown keys in persistent groups and unknown groups
"""
def get_namespace(self, options: Values | Config[Values], defaults: bool = True) -> Config[Namespace] | Namespace:
if isinstance(options, Config):
self.definitions = options[1]
return get_namespace(options, defaults=defaults)
else:
return get_namespace(Config(options, self.definitions), defaults=defaults)
return get_namespace(
self._get_config(config), file=file, cmdline=cmdline, default=default, persistent=persistent,
)
def parse_file(self, filename: pathlib.Path) -> tuple[Config[Values], bool]:
"""
Helper function to read options from a json dictionary from a file.
This is purely a convenience function.
If _anything_ more advanced is desired this should be handled by the application.
Args:
filename: A pathlib.Path object to read a JSON dictionary from
"""
return parse_file(filename=filename, definitions=self.definitions)
def save_file(self, options: T | Config[T], filename: pathlib.Path) -> bool:
if isinstance(options, Config):
return save_file(options, filename=filename)
return save_file(Config(options, self.definitions), filename=filename)
def save_file(self, config: T | Config[T], filename: pathlib.Path) -> bool:
"""
Helper function to save options from a json dictionary to a file.
This is purely a convenience function.
If _anything_ more advanced is desired this should be handled by the application.
def parse_cmdline(self, args: list[str] | None = None, namespace: ns[T] = None) -> Config[Values]:
return parse_cmdline(self.definitions, self.description, self.epilog, args, namespace)
Args:
config: The options to save to a json dictionary
filename: A pathlib.Path object to save the json dictionary to
"""
return save_file(self._get_config(config), filename=filename)
def parse_cmdline(self, args: list[str] | None = None, config: ns[T] = None) -> Config[Values]:
"""
Creates an `argparse.ArgumentParser` from cmdline settings in `self.definitions`.
`args` and `config` are passed to `argparse.ArgumentParser.parse_args`
Args:
args: Passed to argparse.ArgumentParser.parse_args
config: The Config or Namespace object to use as a Namespace passed to argparse.ArgumentParser.parse_args
"""
return parse_cmdline(self.definitions, self.description, self.epilog, args, config)
def parse_config(self, config_path: pathlib.Path, args: list[str] | None = None) -> tuple[Config[Values], bool]:
"""
Convenience function to parse options from a json file and passes the resulting Config object to parse_cmdline.
This is purely a convenience function.
If _anything_ more advanced is desired this should be handled by the application.
Args:
config_path: A `pathlib.Path` object
args: Passed to argparse.ArgumentParser.parse_args
"""
return parse_config(self.definitions, self.description, self.epilog, config_path, args)
def example(manager: Manager) -> None:
__all__ = [
'Setting',
'TypedNS',
'Group',
'Values',
'Definitions',
'Config',
'generate_ns',
'sanitize_name',
'get_option',
'get_options',
'normalize_config',
'parse_file',
'clean_config',
'defaults',
'get_namespace',
'save_file',
'create_argparser',
'parse_cmdline',
'parse_config',
'Manager',
]
def example_group(manager: Manager) -> None:
manager.add_setting(
'--hello',
default='world',
@ -652,7 +925,7 @@ def example(manager: Manager) -> None:
)
def persistent(manager: Manager) -> None:
def persistent_group(manager: Manager) -> None:
manager.add_setting(
'--test', '-t',
default=False,
@ -664,23 +937,23 @@ def _main(args: list[str] | None = None) -> None:
settings_path = pathlib.Path('./settings.json')
manager = Manager(description='This is an example', epilog='goodbye!')
manager.add_group('example', example)
manager.add_persistent_group('persistent', persistent)
manager.add_group('Example Group', example_group)
manager.add_persistent_group('persistent', persistent_group)
file_config, success = manager.parse_file(settings_path)
file_namespace = manager.get_namespace(file_config)
file_namespace = manager.get_namespace(file_config, file=True, cmdline=True)
merged_config = manager.parse_cmdline(args=args, namespace=file_namespace)
merged_namespace = manager.get_namespace(merged_config)
merged_config = manager.parse_cmdline(args=args, config=file_namespace)
merged_namespace = manager.get_namespace(merged_config, file=True, cmdline=True)
print(f'Hello {merged_config.values["example"]["hello"]}') # noqa: T201
if merged_namespace.values.example_save:
print(f'Hello {merged_config.values["Example Group"]["hello"]}') # noqa: T201
if merged_namespace.values.Example_Group__save:
if manager.save_file(merged_config, settings_path):
print(f'Successfully saved settings to {settings_path}') # noqa: T201
else:
else: # pragma: no cover
print(f'Failed saving settings to a {settings_path}') # noqa: T201
if merged_namespace.values.example_verbose:
print(f'{merged_namespace.values.example_verbose=}') # noqa: T201
if merged_namespace.values.Example_Group__verbose:
print(f'{merged_namespace.values.Example_Group__verbose=}') # noqa: T201
if __name__ == '__main__':

View File

@ -1,6 +1,5 @@
[metadata]
name = settngs
version = 0.6.3
description = A library for managing settings
long_description = file: README.md
long_description_content_type = text/markdown
@ -19,7 +18,7 @@ classifiers =
[options]
packages = find:
install_requires =
typing-extensions;python_version < '3.11'
typing-extensions>=4.3.0;python_version < '3.11'
python_requires = >=3.8
include_package_data = True
@ -79,6 +78,8 @@ max_line_length = 120
[flake8]
extend-ignore = E501, A003
max_line_length = 120
per-file-ignores =
*_test.py: LN001
[coverage:run]
plugins = covdefaults
@ -89,6 +90,7 @@ fail_under = 95
[mypy]
check_untyped_defs = true
disallow_any_generics = true
warn_return_any = true
disallow_incomplete_defs = true
disallow_untyped_defs = true
no_implicit_optional = true
@ -96,7 +98,9 @@ warn_redundant_casts = true
warn_unused_ignores = true
[mypy-testing.*]
warn_return_any = false
disallow_untyped_defs = false
[mypy-tests.*]
warn_return_any = false
disallow_untyped_defs = false

View File

@ -15,52 +15,52 @@ example: list[tuple[list[str], str, str]] = [
(
['--hello', 'lordwelch', '-s'],
'Hello lordwelch\nSuccessfully saved settings to settings.json\n',
'{\n "example": {\n "hello": "lordwelch",\n "verbose": false\n },\n "persistent": {\n "test": false\n }\n}\n',
'{\n "Example Group": {\n "hello": "lordwelch",\n "verbose": false\n },\n "persistent": {\n "test": false\n }\n}\n',
),
(
[],
'Hello lordwelch\n',
'{\n "example": {\n "hello": "lordwelch",\n "verbose": false\n },\n "persistent": {\n "test": false\n }\n}\n',
'{\n "Example Group": {\n "hello": "lordwelch",\n "verbose": false\n },\n "persistent": {\n "test": false\n }\n}\n',
),
(
['-v'],
'Hello lordwelch\nmerged_namespace.values.example_verbose=True\n',
'{\n "example": {\n "hello": "lordwelch",\n "verbose": false\n },\n "persistent": {\n "test": false\n }\n}\n',
'Hello lordwelch\nmerged_namespace.values.Example_Group__verbose=True\n',
'{\n "Example Group": {\n "hello": "lordwelch",\n "verbose": false\n },\n "persistent": {\n "test": false\n }\n}\n',
),
(
['-v', '-s'],
'Hello lordwelch\nSuccessfully saved settings to settings.json\nmerged_namespace.values.example_verbose=True\n',
'{\n "example": {\n "hello": "lordwelch",\n "verbose": true\n },\n "persistent": {\n "test": false\n }\n}\n',
'Hello lordwelch\nSuccessfully saved settings to settings.json\nmerged_namespace.values.Example_Group__verbose=True\n',
'{\n "Example Group": {\n "hello": "lordwelch",\n "verbose": true\n },\n "persistent": {\n "test": false\n }\n}\n',
),
(
[],
'Hello lordwelch\nmerged_namespace.values.example_verbose=True\n',
'{\n "example": {\n "hello": "lordwelch",\n "verbose": true\n },\n "persistent": {\n "test": false\n }\n}\n',
'Hello lordwelch\nmerged_namespace.values.Example_Group__verbose=True\n',
'{\n "Example Group": {\n "hello": "lordwelch",\n "verbose": true\n },\n "persistent": {\n "test": false\n }\n}\n',
),
(
['manual settings.json'],
'Hello lordwelch\nmerged_namespace.values.example_verbose=True\n',
'{\n "example": {\n "hello": "lordwelch",\n "verbose": true\n },\n "persistent": {\n "test": false,\n "hello": "world"\n }\n}\n',
'Hello lordwelch\nmerged_namespace.values.Example_Group__verbose=True\n',
'{\n "Example Group": {\n "hello": "lordwelch",\n "verbose": true\n },\n "persistent": {\n "test": false,\n "hello": "world"\n }\n}\n',
),
(
['--no-verbose', '-t'],
'Hello lordwelch\n',
'{\n "example": {\n "hello": "lordwelch",\n "verbose": true\n },\n "persistent": {\n "test": false,\n "hello": "world"\n }\n}\n',
'{\n "Example Group": {\n "hello": "lordwelch",\n "verbose": true\n },\n "persistent": {\n "test": false,\n "hello": "world"\n }\n}\n',
),
(
['--no-verbose', '-s', '-t'],
'Hello lordwelch\nSuccessfully saved settings to settings.json\n',
'{\n "example": {\n "hello": "lordwelch",\n "verbose": false\n },\n "persistent": {\n "test": true,\n "hello": "world"\n }\n}\n',
'{\n "Example Group": {\n "hello": "lordwelch",\n "verbose": false\n },\n "persistent": {\n "test": true,\n "hello": "world"\n }\n}\n',
),
(
['--hello', 'world', '--no-verbose', '--no-test', '-s'],
'Hello world\nSuccessfully saved settings to settings.json\n',
'{\n "example": {\n "hello": "world",\n "verbose": false\n },\n "persistent": {\n "test": false,\n "hello": "world"\n }\n}\n',
'{\n "Example Group": {\n "hello": "world",\n "verbose": false\n },\n "persistent": {\n "test": false,\n "hello": "world"\n }\n}\n',
),
(
[],
'Hello world\n',
'{\n "example": {\n "hello": "world",\n "verbose": false\n },\n "persistent": {\n "test": false,\n "hello": "world"\n }\n}\n',
'{\n "Example Group": {\n "hello": "world",\n "verbose": false\n },\n "persistent": {\n "test": false,\n "hello": "world"\n }\n}\n',
),
]
success = [
@ -77,13 +77,15 @@ success = [
'cmdline': True,
'const': None,
'default': None,
'setting_name': 'test_setting', # dest is calculated by Setting and is not used by argparse
'dest': 'test_setting', # dest is calculated by Setting and is not used by argparse
'display_name': 'test_setting', # defaults to dest
'exclusive': False,
'file': True,
'flag': True,
'group': 'tst',
'help': None,
'internal_name': 'tst_test_setting', # Should almost always be "{group}_{dest}"
'internal_name': 'tst__test_setting', # Should almost always be "{group}__{dest}"
'metavar': 'TEST_SETTING', # Set manually so argparse doesn't use TST_TEST
'nargs': None,
'required': None,
@ -94,7 +96,7 @@ success = [
'choices': None,
'const': None,
'default': None,
'dest': 'tst_test_setting',
'dest': 'tst__test_setting',
'help': None,
'metavar': 'TEST_SETTING',
'nargs': None,
@ -117,13 +119,15 @@ success = [
'cmdline': True,
'const': None,
'default': None,
'setting_name': 'test', # setting_name is calculated by Setting and is not used by argparse
'dest': 'testing', # dest is calculated by Setting and is not used by argparse
'display_name': 'testing', # defaults to dest
'exclusive': False,
'file': True,
'flag': True,
'group': 'tst',
'help': None,
'internal_name': 'tst_testing', # Should almost always be "{group}_{dest}"
'internal_name': 'tst__testing', # Should almost always be "{group}__{dest}"
'metavar': 'TESTING', # Set manually so argparse doesn't use TST_TEST
'nargs': None,
'required': None,
@ -134,7 +138,7 @@ success = [
'choices': None,
'const': None,
'default': None,
'dest': 'tst_testing',
'dest': 'tst__testing',
'help': None,
'metavar': 'TESTING',
'nargs': None,
@ -156,13 +160,15 @@ success = [
'cmdline': True,
'const': None,
'default': None,
'setting_name': 'test', # dest is calculated by Setting and is not used by argparse
'dest': 'test', # dest is calculated by Setting and is not used by argparse
'display_name': 'test', # defaults to dest
'exclusive': False,
'file': True,
'flag': True,
'group': 'tst',
'help': None,
'internal_name': 'tst_test', # Should almost always be "{group}_{dest}"
'internal_name': 'tst__test', # Should almost always be "{group}__{dest}"
'metavar': 'TEST', # Set manually so argparse doesn't use TST_TEST
'nargs': None,
'required': None,
@ -173,7 +179,7 @@ success = [
'choices': None,
'const': None,
'default': None,
'dest': 'tst_test',
'dest': 'tst__test',
'help': None,
'metavar': 'TEST',
'nargs': None,
@ -196,13 +202,15 @@ success = [
'cmdline': True,
'const': None,
'default': None,
'setting_name': 'test', # dest is calculated by Setting and is not used by argparse
'dest': 'test', # dest is calculated by Setting and is not used by argparse
'display_name': 'test', # defaults to dest
'exclusive': False,
'file': True,
'flag': True,
'group': 'tst',
'help': None,
'internal_name': 'tst_test', # Should almost always be "{group}_{dest}"
'internal_name': 'tst__test', # Should almost always be "{group}__{dest}"
'metavar': None, # store_true does not get a metavar
'nargs': None,
'required': None,
@ -213,7 +221,7 @@ success = [
'choices': None,
'const': None,
'default': None,
'dest': 'tst_test',
'dest': 'tst__test',
'help': None,
'metavar': None,
'nargs': None,
@ -235,13 +243,15 @@ success = [
'cmdline': True,
'const': None,
'default': None,
'setting_name': 'test',
'dest': 'test',
'display_name': 'test', # defaults to dest
'exclusive': False,
'file': True,
'flag': True,
'group': 'tst',
'help': None,
'internal_name': 'tst_test',
'internal_name': 'tst__test',
'metavar': 'TEST',
'nargs': None,
'required': None,
@ -252,7 +262,7 @@ success = [
'choices': None,
'const': None,
'default': None,
'dest': 'tst_test',
'dest': 'tst__test',
'help': None,
'metavar': 'TEST',
'nargs': None,
@ -274,18 +284,20 @@ success = [
'cmdline': True,
'const': None,
'default': None,
'setting_name': 'test',
'dest': 'test',
'display_name': 'test', # defaults to dest
'exclusive': False,
'file': True,
'flag': False,
'group': 'tst',
'help': None,
'internal_name': 'tst_test',
'internal_name': 'tst__test',
'metavar': 'TEST',
'nargs': None,
'required': None,
'type': None,
'argparse_args': ('tst_test',),
'argparse_args': ('tst__test',),
'argparse_kwargs': {
'action': None,
'choices': None,
@ -311,10 +323,12 @@ success = [
'cmdline': True,
'const': None,
'default': None,
'setting_name': 'test',
'dest': 'test',
'display_name': 'test', # defaults to dest
'exclusive': False,
'file': True,
'flag': True,
'group': '',
'help': None,
'internal_name': 'test', # No group, leading _ is stripped

View File

@ -1,8 +1,13 @@
from __future__ import annotations
import argparse
import ast
import json
import pathlib
import sys
from collections import defaultdict
from textwrap import dedent
from typing import Generator
import pytest
@ -13,8 +18,44 @@ from testing.settngs import failure
from testing.settngs import success
if sys.version_info < (3, 9): # pragma: no cover
from typing import List
help_output = '''\
usage: __main__.py [-h] [TEST [TEST ...]]
positional arguments:
TEST
optional arguments:
-h, --help show this help message and exit
'''
else: # pragma: no cover
List = list
help_output = '''\
usage: __main__.py [-h] [TEST ...]
positional arguments:
TEST
optional arguments:
-h, --help show this help message and exit
'''
if sys.version_info >= (3, 10): # pragma: no cover
help_output = '''\
usage: __main__.py [-h] [TEST ...]
positional arguments:
TEST
options:
-h, --help show this help message and exit
'''
@pytest.fixture
def settngs_manager():
def settngs_manager() -> Generator[settngs.Manager, None, None]:
manager = settngs.Manager()
yield manager
@ -53,65 +94,123 @@ def test_add_setting(settngs_manager):
assert settngs_manager.add_setting('--test') is None
def test_get_defaults(settngs_manager):
def test_add_setting_invalid_name(settngs_manager):
with pytest.raises(Exception, match='Cannot use test¥ in a namespace'):
assert settngs_manager.add_setting('--test¥') is None
def test_sub_group(settngs_manager):
with pytest.raises(Exception, match='Sub groups are not allowed'):
settngs_manager.add_group('tst', lambda parser: parser.add_group('tst', lambda parser: parser.add_setting('--test2', default='hello')))
def test_sub_persistent_group(settngs_manager):
with pytest.raises(Exception, match='Sub groups are not allowed'):
settngs_manager.add_persistent_group('tst', lambda parser: parser.add_persistent_group('tst', lambda parser: parser.add_setting('--test2', default='hello')))
def test_redefine_persistent_group(settngs_manager):
settngs_manager.add_group('tst', lambda parser: parser.add_setting('--test2', default='hello'))
with pytest.raises(Exception, match='Group already exists and is not persistent'):
settngs_manager.add_persistent_group('tst', None)
def test_exclusive_group(settngs_manager):
settngs_manager.add_group('tst', lambda parser: parser.add_setting('--test', default='hello'), exclusive_group=True)
settngs_manager.create_argparser()
args = settngs_manager.argparser.parse_args(['--test', 'never'])
assert args.tst__test == 'never'
with pytest.raises(SystemExit):
settngs_manager.add_group('tst', lambda parser: parser.add_setting('--test2', default='hello'), exclusive_group=True)
settngs_manager.create_argparser()
args = settngs_manager.argparser.parse_args(['--test', 'never', '--test2', 'never'])
def test_files_group(capsys, settngs_manager):
settngs_manager.add_group('runtime', lambda parser: parser.add_setting('test', default='hello', nargs='*'))
settngs_manager.create_argparser()
settngs_manager.argparser.print_help()
captured = capsys.readouterr()
assert captured.out == help_output
def test_setting_without_group(capsys, settngs_manager):
settngs_manager.add_setting('test', default='hello', nargs='*')
settngs_manager.create_argparser()
settngs_manager.argparser.print_help()
captured = capsys.readouterr()
assert captured.out == help_output
class TestValues:
def test_invalid_normalize(self, settngs_manager):
with pytest.raises(ValueError) as excinfo:
settngs_manager.add_setting('--test', default='hello')
defaults, _ = settngs_manager.normalize_config({}, file=False, cmdline=False)
assert str(excinfo.value) == 'Invalid parameters: you must set either file or cmdline to True'
def test_get_defaults(self, settngs_manager):
settngs_manager.add_setting('--test', default='hello')
defaults, _ = settngs_manager.defaults()
assert defaults['']['test'] == 'hello'
def test_get_defaults_namespace(settngs_manager):
settngs_manager.add_setting('--test', default='hello')
defaults, _ = settngs_manager.get_namespace(settngs_manager.defaults())
assert defaults.test == 'hello'
def test_get_namespace_with_namespace(settngs_manager):
settngs_manager.add_setting('--test', default='hello')
defaults, _ = settngs_manager.get_namespace(argparse.Namespace(test='success'))
assert defaults.test == 'success'
def test_get_defaults_group(settngs_manager):
def test_get_defaults_group(self, settngs_manager):
settngs_manager.add_group('tst', lambda parser: parser.add_setting('--test', default='hello'))
defaults, _ = settngs_manager.defaults()
assert defaults['tst']['test'] == 'hello'
def test_get_defaults_group_space(self, settngs_manager):
settngs_manager.add_group('Testing tst', lambda parser: parser.add_setting('--test', default='hello'))
defaults, _ = settngs_manager.defaults()
assert defaults['Testing tst']['test'] == 'hello'
def test_get_namespace_group(settngs_manager):
settngs_manager.add_group('tst', lambda parser: parser.add_setting('--test', default='hello'))
defaults, _ = settngs_manager.get_namespace(settngs_manager.defaults())
assert defaults.tst_test == 'hello'
def test_cmdline_only(settngs_manager):
def test_cmdline_only(self, settngs_manager):
settngs_manager.add_group('tst', lambda parser: parser.add_setting('--test', default='hello', file=False))
settngs_manager.add_group('tst2', lambda parser: parser.add_setting('--test2', default='hello', cmdline=False))
file_normalized, _ = settngs_manager.normalize_config(settngs_manager.defaults(), file=True)
cmdline_normalized, _ = settngs_manager.normalize_config(settngs_manager.defaults(), cmdline=True)
assert 'test' in cmdline_normalized['tst']
assert 'test2' not in cmdline_normalized['tst2']
assert 'test' not in file_normalized['tst'] # cmdline option not in normalized config
assert 'test2' in file_normalized['tst2'] # file option in normalized config
assert 'test' not in file_normalized['tst']
assert 'test2' in file_normalized['tst2']
assert 'test' in cmdline_normalized['tst'] # cmdline option in normalized config
assert 'test2' not in cmdline_normalized['tst2'] # file option not in normalized config
def test_cmdline_only_persistent_group(settngs_manager):
def test_cmdline_only_persistent_group(self, settngs_manager):
settngs_manager.add_persistent_group('tst', lambda parser: parser.add_setting('--test', default='hello', file=False))
settngs_manager.add_group('tst2', lambda parser: parser.add_setting('--test2', default='hello', cmdline=False))
file_normalized, _ = settngs_manager.normalize_config(settngs_manager.defaults(), file=True)
cmdline_normalized, _ = settngs_manager.normalize_config(settngs_manager.defaults(), cmdline=True)
assert 'test' in cmdline_normalized['tst']
assert 'test2' not in cmdline_normalized['tst2']
assert 'test' not in file_normalized['tst']
assert 'test2' in file_normalized['tst2']
assert 'test' in cmdline_normalized['tst']
assert 'test2' not in cmdline_normalized['tst2']
def test_normalize(settngs_manager):
def test_normalize_defaults(self, settngs_manager):
settngs_manager.add_group('tst', lambda parser: parser.add_setting('--test', default='hello'))
settngs_manager.add_group('tst', lambda parser: parser.add_setting('--test2', default='hello'))
settngs_manager.add_persistent_group('tst_persistent', lambda parser: parser.add_setting('--test', default='hello'))
defaults = settngs_manager.defaults()
defaults_normalized = settngs_manager.normalize_config(defaults, file=True, default=False)
assert defaults_normalized.values['tst'] == {}
assert defaults_normalized.values['tst_persistent'] == {}
non_defaults = settngs_manager.defaults()
non_defaults.values['tst']['test'] = 'world'
non_defaults.values['tst_persistent']['test'] = 'world'
non_defaults_normalized = settngs_manager.normalize_config(non_defaults, file=True, default=False)
assert non_defaults_normalized.values['tst'] == {'test': 'world'}
assert non_defaults_normalized.values['tst_persistent'] == {'test': 'world'}
def test_normalize(self, settngs_manager):
settngs_manager.add_group('tst', lambda parser: parser.add_setting('--test', default='hello'))
settngs_manager.add_persistent_group('persistent', lambda parser: parser.add_setting('--world', default='world'))
@ -119,13 +218,7 @@ def test_normalize(settngs_manager):
defaults.values['test'] = 'fail' # Not defined in settngs_manager, should be removed
defaults.values['persistent']['hello'] = 'success' # Not defined in settngs_manager, should stay
defaults_namespace = settngs_manager.get_namespace(settngs_manager.defaults())
defaults_namespace.values.test = 'fail' # Not defined in settngs_manager, should be removed
defaults_namespace.values.persistent_hello = 'success' # Not defined in settngs_manager, should stay
normalized, _ = settngs_manager.normalize_config(defaults, file=True)
normalized_from_namespace = settngs_manager.normalize_config(defaults_namespace, file=True)
normalized_namespace, _ = settngs_manager.get_namespace(normalized_from_namespace)
assert 'test' not in normalized
assert 'tst' in normalized
@ -134,11 +227,145 @@ def test_normalize(settngs_manager):
assert normalized['persistent']['hello'] == 'success'
assert normalized['persistent']['world'] == 'world'
assert not hasattr(normalized_namespace, 'test')
assert hasattr(normalized_namespace, 'tst_test')
assert normalized_namespace.tst_test == 'hello'
assert normalized_namespace.persistent_hello == 'success'
assert normalized_namespace.persistent_world == 'world'
def test_unknown_group(self):
manager = settngs.Manager()
manager.add_group('tst', lambda parser: parser.add_setting('--test', default='hello'))
manager.add_persistent_group('persistent', lambda parser: parser.add_setting('--world', default='world'))
manager_unknown = settngs.Manager()
manager_unknown.add_group('tst', lambda parser: parser.add_setting('--test', default='hello'))
# This manager doesn't know about this group
# manager_unknown.add_persistent_group('persistent', lambda parser: parser.add_setting('--world', default='world'))
defaults = manager.defaults()
defaults.values['test'] = 'fail' # type: ignore[assignment] # Not defined in manager, should be removed
defaults.values['persistent']['hello'] = 'success' # Group is not defined in manager_unknown, should stay
normalized, _ = manager_unknown.normalize_config(defaults.values, file=True)
assert 'test' not in normalized
assert 'tst' in normalized
assert 'test' in normalized['tst']
assert normalized['tst']['test'] == 'hello'
assert normalized['persistent']['hello'] == 'success'
assert normalized['persistent']['world'] == 'world'
class TestNamespace:
def test_invalid_normalize(self, settngs_manager):
with pytest.raises(ValueError) as excinfo:
settngs_manager.add_setting('--test', default='hello')
defaults, _ = settngs_manager.get_namespace(settngs_manager.defaults(), file=False, cmdline=False)
assert str(excinfo.value) == 'Invalid parameters: you must set either file or cmdline to True'
def test_get_defaults(self, settngs_manager):
settngs_manager.add_setting('--test', default='hello')
defaults, _ = settngs_manager.get_namespace(settngs_manager.defaults(), file=True, cmdline=True)
assert defaults.test == 'hello'
def test_get_defaults_group(self, settngs_manager):
settngs_manager.add_group('tst', lambda parser: parser.add_setting('--test', default='hello'))
defaults, _ = settngs_manager.get_namespace(settngs_manager.defaults(), file=True, cmdline=True)
assert defaults.tst__test == 'hello'
def test_get_defaults_group_space(self, settngs_manager):
settngs_manager.add_group('Testing tst', lambda parser: parser.add_setting('--test', default='hello'))
defaults, _ = settngs_manager.get_namespace(settngs_manager.defaults(), file=True, cmdline=True)
assert defaults.Testing_tst__test == 'hello'
def test_cmdline_only(self, settngs_manager):
settngs_manager.add_group('tst', lambda parser: parser.add_setting('--test', default='hello', file=False))
settngs_manager.add_group('tst2', lambda parser: parser.add_setting('--test2', default='hello', cmdline=False))
file_normalized, _ = settngs_manager.get_namespace(settngs_manager.normalize_config(settngs_manager.defaults(), file=True), file=True)
cmdline_normalized, _ = settngs_manager.get_namespace(settngs_manager.normalize_config(settngs_manager.defaults(), cmdline=True), cmdline=True)
assert 'tst__test' not in file_normalized.__dict__
assert 'tst2__test2' in file_normalized.__dict__
assert 'tst__test' in cmdline_normalized.__dict__
assert 'tst2__test2' not in cmdline_normalized.__dict__
def test_cmdline_only_persistent_group(self, settngs_manager):
settngs_manager.add_persistent_group('tst', lambda parser: parser.add_setting('--test', default='hello', file=False))
settngs_manager.add_group('tst2', lambda parser: parser.add_setting('--test2', default='hello', cmdline=False))
file_normalized, _ = settngs_manager.get_namespace(settngs_manager.normalize_config(settngs_manager.defaults(), file=True), file=True)
cmdline_normalized, _ = settngs_manager.get_namespace(settngs_manager.normalize_config(settngs_manager.defaults(), cmdline=True), cmdline=True)
assert 'tst__test' not in file_normalized.__dict__
assert 'tst2__test2' in file_normalized.__dict__
assert 'tst__test' in cmdline_normalized.__dict__
assert 'tst2__test2' not in cmdline_normalized.__dict__
def test_normalize_defaults(self, settngs_manager):
settngs_manager.add_group('tst', lambda parser: parser.add_setting('--test', default='hello'))
settngs_manager.add_group('tst', lambda parser: parser.add_setting('--test2', default='hello'))
settngs_manager.add_persistent_group('tst_persistent', lambda parser: parser.add_setting('--test', default='hello'))
defaults = settngs_manager.defaults()
defaults_normalized = settngs_manager.get_namespace(settngs_manager.normalize_config(defaults, file=True, default=False), file=True, default=False)
assert defaults_normalized.values.__dict__ == {}
non_defaults = settngs_manager.get_namespace(settngs_manager.defaults(), file=True, cmdline=True)
non_defaults.values.tst__test = 'world'
non_defaults.values.tst_persistent__test = 'world'
non_defaults_normalized = settngs_manager.get_namespace(settngs_manager.normalize_config(non_defaults, file=True, default=False), file=True, default=False)
assert non_defaults_normalized.values.tst__test == 'world'
assert non_defaults_normalized.values.tst_persistent__test == 'world'
def test_normalize(self, settngs_manager):
settngs_manager.add_group('tst', lambda parser: parser.add_setting('--test', default='hello'))
settngs_manager.add_persistent_group('persistent', lambda parser: parser.add_setting('--world', default='world'))
defaults = settngs_manager.get_namespace(settngs_manager.defaults(), file=True, cmdline=True)
defaults.values.test = 'fail' # Not defined in settngs_manager, should be removed
defaults.values.persistent__hello = 'success' # Not defined in settngs_manager, should stay
normalized, _ = settngs_manager.get_namespace(settngs_manager.normalize_config(defaults, file=True), file=True)
assert not hasattr(normalized, 'test')
assert hasattr(normalized, 'tst__test')
assert normalized.tst__test == 'hello'
assert normalized.persistent__hello == 'success'
assert normalized.persistent__world == 'world'
def test_normalize_unknown(self, settngs_manager):
manager = settngs.Manager()
manager.add_group('tst', lambda parser: parser.add_setting('--test', default='hello'))
manager.add_persistent_group('persistent', lambda parser: parser.add_setting('--world', default='world'))
manager_unknown = settngs.Manager()
manager_unknown.add_group('tst', lambda parser: parser.add_setting('--test', default='hello'))
# This manager doesn't know about this group
# manager_unknown.add_persistent_group('persistent', lambda parser: parser.add_setting('--world', default='world'))
defaults = manager.get_namespace(manager.defaults(), file=True, cmdline=True)
defaults.values.test = 'fail' # Not defined in manager, should be removed
defaults.values.persistent__hello = 'success' # Not defined in manager, should stay
normalized, _ = manager_unknown.get_namespace(defaults.values, file=True)
assert not hasattr(normalized, 'test')
assert hasattr(normalized, 'tst__test')
assert normalized.tst__test == 'hello'
assert normalized.persistent__hello == 'success'
assert normalized.persistent__world == 'world'
def test_get_namespace_with_namespace(settngs_manager):
settngs_manager.add_setting('--test', default='hello')
defaults, _ = settngs_manager.get_namespace(argparse.Namespace(test='success'), file=True)
assert defaults.test == 'success'
def test_get_namespace_group(settngs_manager):
settngs_manager.add_group('tst', lambda parser: parser.add_setting('--test', default='hello'))
defaults, _ = settngs_manager.get_namespace(settngs_manager.defaults(), file=True)
assert defaults.tst__test == 'hello'
def test_clean_config(settngs_manager):
@ -169,8 +396,8 @@ def test_parse_cmdline(settngs_manager):
namespaces = (
lambda definitions: settngs.Config({'tst': {'test': 'fail', 'test2': 'success'}}, definitions),
lambda definitions: settngs.Config(argparse.Namespace(tst_test='fail', tst_test2='success'), definitions),
lambda definitions: argparse.Namespace(tst_test='fail', tst_test2='success'),
lambda definitions: settngs.Config(argparse.Namespace(tst__test='fail', tst__test2='success'), definitions),
lambda definitions: argparse.Namespace(tst__test='fail', tst__test2='success'),
)
@ -180,7 +407,7 @@ def test_parse_cmdline_with_namespace(settngs_manager, ns):
settngs_manager.add_group('tst', lambda parser: parser.add_setting('--test2', default='fail', cmdline=True))
normalized, _ = settngs_manager.parse_cmdline(
['--test', 'success'], namespace=ns(settngs_manager.definitions),
['--test', 'success'], config=ns(settngs_manager.definitions),
)
assert 'test' in normalized['tst']
@ -331,7 +558,7 @@ def test_adding_to_existing_group(settngs_manager, tmp_path):
assert default_to_regular(settngs_manager.definitions) == default_to_regular(settngs_manager2.definitions)
def test_adding_to_existing_persistent_group(settngs_manager, tmp_path):
def test_adding_to_existing_persistent_group(settngs_manager: settngs.Manager, tmp_path: pathlib.Path) -> None:
def default_to_regular(d):
if isinstance(d, defaultdict):
d = {k: default_to_regular(v) for k, v in d.items()}
@ -349,6 +576,126 @@ def test_adding_to_existing_persistent_group(settngs_manager, tmp_path):
assert default_to_regular(settngs_manager.definitions) == default_to_regular(settngs_manager2.definitions)
class test_type(int):
...
def _typed_function(something: str) -> test_type: # pragma: no cover
return test_type()
def _untyped_function(something):
...
class _customAction(argparse.Action): # pragma: no cover
def __init__(
self,
option_strings,
dest,
const=None,
default=None,
required=False,
help=None, # noqa: A002
metavar=None,
):
super().__init__(
option_strings=option_strings,
dest=dest,
nargs=0,
const=const,
default=default,
required=required,
help=help,
)
def __call__(self, parser, namespace, values, option_string=None):
setattr(namespace, self.dest, 'Something')
types = (
(settngs.Setting('-t', '--test'), str),
(settngs.Setting('-t', '--test', cmdline=False), 'Any'),
(settngs.Setting('-t', '--test', default=1, file=True, cmdline=False), int),
(settngs.Setting('-t', '--test', action='count'), int),
(settngs.Setting('-t', '--test', action='append'), List[str]),
(settngs.Setting('-t', '--test', action='extend'), List[str]),
(settngs.Setting('-t', '--test', action='store_const', const=1), int),
(settngs.Setting('-t', '--test', action='append_const', const=1), list),
(settngs.Setting('-t', '--test', action='store_true'), bool),
(settngs.Setting('-t', '--test', action='store_false'), bool),
(settngs.Setting('-t', '--test', action=settngs.BooleanOptionalAction), bool),
(settngs.Setting('-t', '--test', action=_customAction), 'Any'),
(settngs.Setting('-t', '--test', action='help'), None),
(settngs.Setting('-t', '--test', action='version'), None),
(settngs.Setting('-t', '--test', type=int), int),
(settngs.Setting('-t', '--test', type=_typed_function), test_type),
(settngs.Setting('-t', '--test', type=_untyped_function, default=1), int),
(settngs.Setting('-t', '--test', type=_untyped_function), 'Any'),
)
@pytest.mark.parametrize('setting,typ', types)
def test_guess_type(setting, typ):
guessed_type = setting._guess_type()
assert guessed_type == typ
settings = (
(lambda parser: parser.add_setting('-t', '--test'), 'str'),
(lambda parser: parser.add_setting('-t', '--test', cmdline=False), 'typing.Any'),
(lambda parser: parser.add_setting('-t', '--test', default=1, file=True, cmdline=False), 'int'),
(lambda parser: parser.add_setting('-t', '--test', action='count'), 'int'),
(lambda parser: parser.add_setting('-t', '--test', action='append'), List[str]),
(lambda parser: parser.add_setting('-t', '--test', action='extend'), List[str]),
(lambda parser: parser.add_setting('-t', '--test', action='store_const', const=1), 'int'),
(lambda parser: parser.add_setting('-t', '--test', action='append_const', const=1), 'list'),
(lambda parser: parser.add_setting('-t', '--test', action='store_true'), 'bool'),
(lambda parser: parser.add_setting('-t', '--test', action='store_false'), 'bool'),
(lambda parser: parser.add_setting('-t', '--test', action=settngs.BooleanOptionalAction), 'bool'),
(lambda parser: parser.add_setting('-t', '--test', action=_customAction), 'typing.Any'),
(lambda parser: parser.add_setting('-t', '--test', action='help'), None),
(lambda parser: parser.add_setting('-t', '--test', action='version'), None),
(lambda parser: parser.add_setting('-t', '--test', type=int), 'int'),
(lambda parser: parser.add_setting('-t', '--test', nargs='+'), List[str]),
(lambda parser: parser.add_setting('-t', '--test', type=_typed_function), 'tests.settngs_test.test_type'),
(lambda parser: parser.add_setting('-t', '--test', type=_untyped_function, default=1), 'int'),
(lambda parser: parser.add_setting('-t', '--test', type=_untyped_function), 'typing.Any'),
)
@pytest.mark.parametrize('set_options,typ', settings)
def test_generate_ns(settngs_manager, set_options, typ):
settngs_manager.add_group('test', set_options)
src = dedent('''\
from __future__ import annotations
import settngs
''')
if 'typing.' in str(typ):
src += '\nimport typing'
if typ == 'tests.settngs_test.test_type':
src += '\nimport tests.settngs_test'
src += dedent('''
class settngs_namespace(settngs.TypedNS):
''')
if typ is None:
src += ' ...\n'
else:
src += f' {settngs_manager.definitions["test"].v["test"].internal_name}: {typ}\n'
generated_src = settngs_manager.generate_ns()
assert generated_src == src
ast.parse(generated_src)
def test_example(capsys, tmp_path, monkeypatch):
monkeypatch.chdir(tmp_path)
settings_file = tmp_path / 'settings.json'
@ -358,7 +705,9 @@ def test_example(capsys, tmp_path, monkeypatch):
for args, expected_out, expected_file in example:
if args == ['manual settings.json']:
settings_file.unlink()
settings_file.write_text('{\n "example": {\n "hello": "lordwelch",\n "verbose": true\n },\n "persistent": {\n "test": false,\n "hello": "world"\n }\n}\n')
settings_file.write_text('{\n "Example Group": {\n "hello": "lordwelch",\n "verbose": true\n },\n "persistent": {\n "test": false,\n "hello": "world"\n }\n}\n')
i += 1
continue
else:
settngs._main(args)
captured = capsys.readouterr()