Merge branch 'filenameParser' into develop

I created a new, mostly over complicated, filename parser
The new parser works well in many cases and will collect more data than
the original parser but will sometimes give odd results because of how
complicated it has been made e.g.
'100 page giant' will cause issues however '100-page giant' will not

Remove the parse scan info setting as it was not respected in many cases

.flake8

@@ -1,4 +1,4 @@
 [flake8]
 max-line-length = 120
 extend-ignore = E203, E501, E722
-extend-exclude = venv, scripts
+extend-exclude = venv, scripts, build, dist

CONTRIBUTING.md

@@ -0,0 +1,137 @@
+# How to contribute
+
+If your not sure what you can do or you need to ask a question or just want to talk about ComicTagger head over to the [discussions tab](https://github.com/comictagger/comictagger/discussions/categories/general) and start a discussion
+
+## Tests
+
+We have tests written using pytest! Some of them even pass! If you are contributing code any tests you can write are appreciated.
+
+A great place to start is extending the tests that are already made.
+
+For example the file tests/filenames.py has lists of filenames to be parsed in the format:
+```py
+    pytest.param(
+        "Star Wars - War of the Bounty Hunters - IG-88 (2021) (Digital) (Kileko-Empire).cbz",
+        "number ends series, no-issue",
+        {
+            "issue": "",
+            "series": "Star Wars - War of the Bounty Hunters - IG-88",
+            "volume": "",
+            "year": "2021",
+            "remainder": "(Digital) (Kileko-Empire)",
+            "issue_count": "",
+        },
+        marks=pytest.mark.xfail,
+    )
+```
+
+A test consists of 3-4 parts
+1. The filename to be parsed
+2. The reason it might fail
+3. What the result of parsing the filename should be
+4. `marks=pytest.mark.xfail` This marks the test as expected to fail
+
+If you are not comfortable creating a pull request you can [open an issue](https://github.com/comictagger/comictagger/issues/new/choose) or [start a discussion](https://github.com/comictagger/comictagger/discussions/new)
+
+## Submitting changes
+
+Please open a [GitHub Pull Request](https://github.com/comictagger/comictagger/pull/new/develop) with a clear list of what you've done (read more about [pull requests](http://help.github.com/pull-requests/)). When you send a pull request, we will love you forever if you include tests. We can always use more test coverage. Please run the code tools below and make sure all of your commits are atomic (one feature per commit).
+
+## Contributing Code
+
+Currently only python 3.9 is supported however 3.10 will probably work if you try it
+
+Those on linux should install `Pillow` from the system package manager if possible and if the GUI and/or the CBR/RAR comicbooks are going to be used `pyqt5` and `unrar-cffi` should be installed from the system package manager
+
+Those on macOS will need to ensure that you are using python3 in x86 mode either by installing an x86 only version of python or using the universal installer and using `python3-intel64` instead of `python3`
+
+1. Clone the repository
+```
+git clone https://github.com/comictagger/comictagger.git
+```
+
+2. It is preferred to use a virtual env for running from source, adding the `--system-site-packages` allows packages already installed via the system package manager to be used:
+
+```
+python3 -m venv --system-site-packages venv
+```
+
+3. Activate the virtual env:
+```
+. venv/bin/activate
+```
+or if on windows PowerShell
+```
+. venv/bin/activate.ps1
+```
+
+4. install dependencies:
+```bash
+pip install -r requirements_dev.txt -r requirements.txt
+# if installing optional dependencies
+pip install -r requirements-GUI.txt -r requirements-CBR.txt
+```
+
+5. install ComicTagger
+```
+pip install .
+```
+
+6. (optionall) run pytest to ensure that their are no failures (xfailed means expected failure)
+```
+$ pytest
+============================= test session starts ==============================
+platform darwin -- Python 3.9.12, pytest-7.1.1, pluggy-1.0.0
+rootdir: /Users/timmy/build/source/comictagger
+collected 61 items
+
+tests/test_FilenameParser.py ..x......x.xxx.xx....xxxxxx.xx.x..xxxxxxx   [ 67%]
+tests/test_comicarchive.py x...                                          [ 73%]
+tests/test_rename.py ..xxx.xx..XXX.XX                                    [100%]
+
+================== 27 passed, 29 xfailed, 5 xpassed in 2.68s ===================
+```
+
+7. Make your changes
+8. run code tools and correct any issues
+```bash
+black .
+isort .
+flake8 .
+pytest
+```
+
+black: formats all of the code consistently so there are no surprises<br>
+isort: sorts imports so that you can always find where an import is located<br>
+flake8: checks for code quality and style (warns for unused imports and similar issues)<br>
+pytest: runs tests for ComicTagger functionality
+
+
+if on mac or linux most of this can be accomplished by running
+```
+make install
+# or make PYTHON=python3-intel64 install
+. venv/bin/activate
+make CI
+```
+There is also `make check` which will run all of the code tools in a read-only capacity
+```
+$ make check
+venv/bin/black --check .
+All done! ✨ 🍰 ✨
+52 files would be left unchanged.
+venv/bin/isort --check .
+Skipped 6 files
+venv/bin/flake8 .
+venv/bin/pytest
+============================= test session starts ==============================
+platform darwin -- Python 3.9.12, pytest-7.1.1, pluggy-1.0.0
+rootdir: /Users/timmy/build/source/comictagger
+collected 61 items
+
+tests/test_FilenameParser.py ..x......x.xxx.xx....xxxxxx.xx.x..xxxxxxx   [ 67%]
+tests/test_comicarchive.py x...                                          [ 73%]
+tests/test_rename.py ..xxx.xx..XXX.XX                                    [100%]
+
+================== 27 passed, 29 xfailed, 5 xpassed in 2.68s ===================
+```

Makefile

@@ -2,6 +2,16 @@ PIP ?= pip3
 PYTHON ?= python3
 VERSION_STR := $(shell $(PYTHON) setup.py --version)
 
+SITE_PACKAGES := $(shell $(PYTHON) -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])')
+PACKAGE_PATH = $(SITE_PACKAGES)/comictagger-$(VERSION_STR).dist-info
+
+VENV := $(shell echo $${VIRTUAL_ENV-venv})
+PY3 := $(shell command -v $(PYTHON) 2> /dev/null)
+PYTHON_VENV := $(VENV)/bin/python
+INSTALL_STAMP := $(VENV)/.install.stamp
+INSTALL_GUI_STAMP := $(VENV)/.install-GUI.stamp
+
+
 ifeq ($(OS),Windows_NT)
 	OS_VERSION=win-$(PROCESSOR_ARCHITECTURE)
 	APP_NAME=comictagger.exe
@@ -15,26 +25,33 @@ else
 	FINAL_NAME=ComicTagger-$(VERSION_STR)
 endif
 
-.PHONY: all clean pydist upload dist CI
+.PHONY: all clean pydist upload dist CI check run
 
 all: clean dist
 
+$(PYTHON_VENV):
+	@if [ -z $(PY3) ]; then echo "Python 3 could not be found."; exit 2; fi
+	$(PY3) -m venv --system-site-packages $(VENV)
+
 clean:
-	rm -rf *~ *.pyc *.pyo
-	rm -rf scripts/*.pyc
-	cd comictaggerlib; rm -f *~ *.pyc *.pyo
+	find . -type d -name "__pycache__" | xargs rm -rf {};
+	rm -rf $(INSTALL_STAMP)
 	rm -rf dist MANIFEST
-	rm -rf *.deb
-	rm -rf logdict*.log
 	$(MAKE) -C mac clean
 	rm -rf build
-	rm -rf comictaggerlib/ui/__pycache__
 	rm comictaggerlib/ctversion.py
 
-CI:
+CI: ins
 	black .
 	isort .
 	flake8 .
+	pytest
+
+check: install
+	$(VENV)/bin/black --check .
+	$(VENV)/bin/isort --check .
+	$(VENV)/bin/flake8 .
+	$(VENV)/bin/pytest
 
 pydist: CI
 	make clean
@@ -48,7 +65,21 @@ upload:
 	$(PYTHON) setup.py register
 	$(PYTHON) setup.py sdist --formats=gztar upload
 
+install: $(INSTALL_STAMP)
+$(INSTALL_STAMP): $(PYTHON_VENV) requirements.txt requirements_dev.txt
+	$(PYTHON_VENV) -m pip install -r requirements_dev.txt
+	$(PYTHON_VENV) -m pip install -e .
+	touch $(INSTALL_STAMP)
+
+install-GUI: $(INSTALL_GUI_STAMP)
+$(INSTALL_GUI_STAMP): requirements-GUI.txt
+	$(PYTHON_VENV) -m pip install -r requirements-GUI.txt
+	touch $(INSTALL_GUI_STAMP)
+
+ins: $(PACKAGE_PATH)
+$(PACKAGE_PATH):
+	$(PIP) install -e .
+
 dist: CI
-	$(PIP) install .
 	pyinstaller -y comictagger.spec
 	cd dist && zip -r $(FINAL_NAME).zip $(APP_NAME)

README.md

@@ -1,4 +1,4 @@
-[![Build Status](https://travis-ci.org/comictagger/comictagger.svg?branch=develop)](https://travis-ci.org/comictagger/comictagger)
+[![Build](https://github.com/comictagger/comictagger/actions/workflows/build.yaml/badge.svg)](https://github.com/comictagger/comictagger/actions/workflows/build.yaml)
 [![Gitter chat](https://badges.gitter.im/gitterHQ/gitter.png)](https://gitter.im/comictagger/community)
 [![Google Group](https://img.shields.io/badge/discuss-on%20groups-%23207de5)](https://groups.google.com/forum/#!forum/comictagger)
 [![Twitter](https://img.shields.io/badge/%40comictagger-twitter-lightgrey)](https://twitter.com/comictagger)
@@ -21,7 +21,7 @@ ComicTagger is a **multi-platform** app for **writing metadata to digital comics
 * Native read only support for **CBR** digital comics: full support enabled installing additional [rar tools](https://www.rarlab.com/download.htm)
 * Command line interface (CLI) enabling **custom scripting** and **batch operations on large collections**
 
-For details, screen-shots, release notes, and more, visit [the Wiki](https://github.com/comictagger/comictagger/wiki)
+For details, screen-shots, and more, visit [the Wiki](https://github.com/comictagger/comictagger/wiki)
 
 
 ## Installation
@@ -40,11 +40,11 @@ A pip package is provided, you can install it with:
  $ pip3 install comictagger[GUI]
 ```
 
+There are two optional dependencies GUI and CBR. You can install the optional dependencies by specifying one or more of `GUI`,`CBR` or `all` in braces e.g. `comictagger[CBR,GUI]`
+
 ### From source
 
- 1. Ensure you have a recent version of python3 installed
+ 1. Ensure you have python 3.9 installed
  2. Clone this repository `git clone https://github.com/comictagger/comictagger.git`
  3. `pip3 install -r requirements_dev.txt`
- 4. Optionally install the GUI `pip3 install -r requirements-GUI.txt`
- 5. Optionally install CBR support `pip3 install -r requirements-CBR.txt`
- 6. `python3 comictagger.py`
+ 7. `pip3 install .` or `pip3 install .[GUI]`

comicapi/comet.py

@@ -121,7 +121,7 @@ class CoMet:
             if credit["role"].lower() in set(self.editor_synonyms):
                 ET.SubElement(root, "editor").text = str(credit["person"])
 
-        utils.indent(root)
+        ET.indent(root)
 
         # wrap it in an ElementTree instance, and save as XML
         tree = ET.ElementTree(root)

comicapi/comicarchive.py

@@ -30,8 +30,10 @@ import py7zr
 
 try:
     from unrar.cffi import rarfile
+
+    rar_support = True
 except:
-    pass
+    rar_support = False
 
 try:
     from PIL import Image
@@ -40,10 +42,10 @@ try:
 except ImportError:
     pil_available = False
 
+from comicapi import filenamelexer, filenameparser
 from comicapi.comet import CoMet
 from comicapi.comicbookinfo import ComicBookInfo
 from comicapi.comicinfoxml import ComicInfoXml
-from comicapi.filenameparser import FileNameParser
 from comicapi.genericmetadata import GenericMetadata, PageType
 
 logger = logging.getLogger(__name__)
@@ -81,10 +83,10 @@ class SevenZipArchiver:
                 data = zf.read(archive_file)[archive_file].read()
         except py7zr.Bad7zFile as e:
             logger.error("bad 7zip file [%s]: %s :: %s", e, self.path, archive_file)
-            raise IOError
+            raise IOError from e
         except Exception as e:
             logger.error("bad 7zip file [%s]: %s :: %s", e, self.path, archive_file)
-            raise IOError
+            raise IOError from e
 
         return data
 
@@ -92,6 +94,7 @@ class SevenZipArchiver:
         try:
             self.rebuild_zip_file([archive_file])
         except:
+            logger.exception("Failed to remove %s from 7zip archive", archive_file)
             return False
         else:
             return True
@@ -110,6 +113,7 @@ class SevenZipArchiver:
                 zf.writestr(data, archive_file)
             return True
         except:
+            logger.exception("Writing zip file failed")
             return False
 
     def get_filename_list(self):
@@ -131,15 +135,14 @@ class SevenZipArchiver:
         os.close(tmp_fd)
 
         try:
-            with py7zr.SevenZipFile(self.path, "r") as zip:
-                targets = [f for f in zip.getnames() if f not in exclude_list]
+            with py7zr.SevenZipFile(self.path, "r") as zin:
+                targets = [f for f in zin.getnames() if f not in exclude_list]
             with py7zr.SevenZipFile(self.path, "r") as zin:
                 with py7zr.SevenZipFile(tmp_name, "w") as zout:
                     for fname, bio in zin.read(targets).items():
                         zout.writef(bio, fname)
         except Exception:
             logger.exception("Error rebuilding 7zip file: %s", self.path)
-            return []
 
         # replace with the new file
         os.remove(self.path)
@@ -194,6 +197,7 @@ class ZipArchiver:
         try:
             self.rebuild_zip_file([archive_file])
         except:
+            logger.exception("Failed to remove %s from zip archive", archive_file)
             return False
         else:
             return True
@@ -243,8 +247,7 @@ class ZipArchiver:
                     # preserve the old comment
                     zout.comment = zin.comment
         except Exception:
-            logger.exception("Error rebuilding 7zip file: %s", self.path)
-            return []
+            logger.exception("Error rebuilding zip file: %s", self.path)
 
         # replace with the new file
         os.remove(self.path)
@@ -306,7 +309,7 @@ class ZipArchiver:
                 else:
                     raise Exception("Failed to write comment to zip file!")
         except Exception:
-            logger.exception()
+            logger.exception("Writing comment to %s failed", filename)
             return False
         else:
             return True
@@ -342,7 +345,7 @@ class RarArchiver:
         self.rar_exe_path = rar_exe_path
 
         if RarArchiver.devnull is None:
-            RarArchiver.devnull = open(os.devnull, "w")
+            RarArchiver.devnull = open(os.devnull, "bw")
 
         # windows only, keeps the cmd.exe from popping up
         if platform.system() == "Windows":
@@ -360,9 +363,8 @@ class RarArchiver:
             try:
                 # write comment to temp file
                 tmp_fd, tmp_name = tempfile.mkstemp()
-                f = os.fdopen(tmp_fd, "w+")
-                f.write(comment)
-                f.close()
+                with os.fdopen(tmp_fd, "wb") as f:
+                    f.write(comment.encode("utf-8"))
 
                 working_dir = os.path.dirname(os.path.abspath(self.path))
 
@@ -418,8 +420,7 @@ class RarArchiver:
                 break
 
             else:
-                # Success"
-                # entries is a list of of tuples:  ( rarinfo, filedata)
+                # Success. Entries is a list of of tuples:  ( rarinfo, filedata)
                 if tries > 1:
                     logger.info("Attempted read_files() {%d} times", tries)
                 if len(entries) == 1:
@@ -441,7 +442,7 @@ class RarArchiver:
 
                 # TODO: will this break if 'archive_file' is in a subfolder. i.e. "foo/bar.txt"
                 # will need to create the subfolder above, I guess...
-                with open(tmp_file, "w") as f:
+                with open(tmp_file, "wb") as f:
                     f.write(data)
 
                 # use external program to write file to Rar archive
@@ -457,7 +458,9 @@ class RarArchiver:
                     time.sleep(1)
                 os.remove(tmp_file)
                 os.rmdir(tmp_folder)
-            except:
+            except Exception as e:
+                logger.info(str(e))
+                logger.exception("Failed write %s to rar archive", archive_file)
                 return False
             else:
                 return True
@@ -479,6 +482,7 @@ class RarArchiver:
                 if platform.system() == "Darwin":
                     time.sleep(1)
             except:
+                logger.exception("Failed to remove %s from rar archive", archive_file)
                 return False
             else:
                 return True
@@ -543,7 +547,6 @@ class FolderArchiver:
         try:
             with open(fname, "rb") as f:
                 data = f.read()
-                f.close()
         except IOError:
             logger.exception("Failed to read: %s", fname)
 
@@ -553,11 +556,10 @@ class FolderArchiver:
 
         fname = os.path.join(self.path, archive_file)
         try:
-            with open(fname, "w+") as f:
+            with open(fname, "wb") as f:
                 f.write(data)
-                f.close()
         except:
-            logger.exception("Failed to read: %s", fname)
+            logger.exception("Failed to write: %s", fname)
             return False
         else:
             return True
@@ -568,7 +570,7 @@ class FolderArchiver:
         try:
             os.remove(fname)
         except:
-            logger.exception("Failed to read: %s", fname)
+            logger.exception("Failed to remove: %s", fname)
             return False
         else:
             return True
@@ -588,7 +590,6 @@ class FolderArchiver:
         return itemlist
 
 
-# noinspection PyUnusedLocal
 class UnknownArchiver:
 
     """Unknown implementation"""
@@ -876,7 +877,7 @@ class ComicArchive:
             self.page_list = []
             for name in files:
                 if (
-                    os.path.splitext(name)[1].lower() in [".jpg", "jpeg", ".png", ".gif", ".webp"]
+                    os.path.splitext(name)[1].lower() in [".jpg", ".jpeg", ".png", ".gif", ".webp"]
                     and os.path.basename(name)[0] != "."
                 ):
                     self.page_list.append(name)
@@ -976,7 +977,7 @@ class ComicArchive:
             if raw_cix == "":
                 raw_cix = None
             cix_string = ComicInfoXml().string_from_metadata(metadata, xml=raw_cix)
-            write_success = self.archiver.write_file(self.ci_xml_filename, cix_string)
+            write_success = self.archiver.write_file(self.ci_xml_filename, cix_string.encode("utf-8"))
             if write_success:
                 self.has__cix = True
                 self.cix_md = metadata
@@ -1088,8 +1089,7 @@ class ComicArchive:
                         data = self.archiver.read_file(n)
                     except Exception as e:
                         data = ""
-                        err_msg = f"Error reading in Comet XML for validation!: {e}"
-                        logger.warning(err_msg)
+                        logger.warning("Error reading in Comet XML for validation!: %s", e)
                     if CoMet().validate_string(data):
                         # since we found it, save it!
                         self.comet_filename = n
@@ -1127,25 +1127,46 @@ class ComicArchive:
                         data = self.get_page(idx)
                         p["ImageSize"] = str(len(data))
 
-    def metadata_from_filename(self, parse_scan_info=True):
+    def metadata_from_filename(
+        self, complicated_parser=False, remove_c2c=False, remove_fcbd=False, remove_publisher=False
+    ):
 
         metadata = GenericMetadata()
 
-        fnp = FileNameParser()
-        fnp.parse_filename(self.path)
+        if complicated_parser:
+            lex = filenamelexer.Lex(self.path)
+            p = filenameparser.Parse(
+                lex.items, remove_c2c=remove_c2c, remove_fcbd=remove_fcbd, remove_publisher=remove_publisher
+            )
+            metadata.alternate_number = p.filename_info["alternate"] or None
+            metadata.issue = p.filename_info["issue"] or None
+            metadata.issue_count = p.filename_info["issue_count"] or None
+            metadata.publisher = p.filename_info["publisher"] or None
+            metadata.series = p.filename_info["series"] or None
+            metadata.title = p.filename_info["title"] or None
+            metadata.volume = p.filename_info["volume"] or None
+            metadata.volume_count = p.filename_info["volume_count"] or None
+            metadata.year = p.filename_info["year"] or None
 
-        if fnp.issue != "":
-            metadata.issue = fnp.issue
-        if fnp.series != "":
-            metadata.series = fnp.series
-        if fnp.volume != "":
-            metadata.volume = fnp.volume
-        if fnp.year != "":
-            metadata.year = fnp.year
-        if fnp.issue_count != "":
-            metadata.issue_count = fnp.issue_count
-        if parse_scan_info:
-            if fnp.remainder != "":
+            metadata.scan_info = p.filename_info["remainder"] or None
+            metadata.format = "FCBD" if p.filename_info["fcbd"] else None
+            if p.filename_info["annual"]:
+                metadata.format = "Annual"
+        else:
+            fnp = filenameparser.FileNameParser()
+            fnp.parse_filename(self.path)
+
+            if fnp.issue:
+                metadata.issue = fnp.issue
+            if fnp.series:
+                metadata.series = fnp.series
+            if fnp.volume:
+                metadata.volume = fnp.volume
+            if fnp.year:
+                metadata.year = fnp.year
+            if fnp.issue_count:
+                metadata.issue_count = fnp.issue_count
+            if fnp.remainder:
                 metadata.scan_info = fnp.remainder
 
         metadata.is_empty = False

comicapi/comicbookinfo.py

@@ -119,5 +119,5 @@ class ComicBookInfo:
 
         cbi_container = self.create_json_dictionary(metadata)
 
-        with open(filename, "w") as f:
+        with open(filename, "w", encoding="utf-8") as f:
             f.write(json.dumps(cbi_container, indent=4))

comicapi/comicinfoxml.py

@@ -79,7 +79,7 @@ class ComicInfoXml:
             else:
                 et_entry = root.find(cix_entry)
                 if et_entry is not None:
-                    et_entry.clear()
+                    root.remove(et_entry)
 
         assign("Title", md.title)
         assign("Series", md.series)
@@ -155,6 +155,7 @@ class ComicInfoXml:
         assign("LanguageISO", md.language)
         assign("Format", md.format)
         assign("AgeRating", md.maturity_rating)
+        assign("CommunityRating", md.community_rating)
         assign("BlackAndWhite", "Yes" if md.black_and_white else None)
         assign("Manga", md.manga)
         assign("Characters", md.characters)
@@ -170,10 +171,13 @@ class ComicInfoXml:
             pages_node = ET.SubElement(root, "Pages")
 
         for page_dict in md.pages:
+            page = page_dict
+            if "Image" in page:
+                page["Image"] = str(page["Image"])
             page_node = ET.SubElement(pages_node, "Page")
             page_node.attrib = dict(sorted(page_dict.items()))
 
-        utils.indent(root)
+        ET.indent(root)
 
         # wrap it in an ElementTree instance, and save as XML
         tree = ET.ElementTree(root)
@@ -222,6 +226,7 @@ class ComicInfoXml:
         md.story_arc = utils.xlate(get("StoryArc"))
         md.series_group = utils.xlate(get("SeriesGroup"))
         md.maturity_rating = utils.xlate(get("AgeRating"))
+        md.community_rating = utils.xlate(get("CommunityRating"))
 
         tmp = utils.xlate(get("BlackAndWhite"))
         if tmp is not None and tmp.lower() in ["yes", "true", "1"]:
@@ -251,6 +256,8 @@ class ComicInfoXml:
         pages_node = root.find("Pages")
         if pages_node is not None:
             for page in pages_node:
+                if "Image" in page.attrib:
+                    page.attrib["Image"] = int(page.attrib["Image"])
                 md.pages.append(page.attrib)
 
         md.is_empty = False

comicapi/filenamelexer.py

@@ -0,0 +1,353 @@
+import calendar
+import os
+import unicodedata
+from enum import Enum, auto
+
+
+class ItemType(Enum):
+    Error = auto()  # Error occurred; value is text of error
+    EOF = auto()
+    Text = auto()  # Text
+    LeftParen = auto()  # '(' inside action
+    Number = auto()  # Simple number
+    IssueNumber = auto()  # Preceded by a # Symbol
+    RightParen = auto()  # ')' inside action
+    Space = auto()  # Run of spaces separating arguments
+    Dot = auto()
+    LeftBrace = auto()
+    RightBrace = auto()
+    LeftSBrace = auto()
+    RightSBrace = auto()
+    Symbol = auto()
+    Skip = auto()  # __ or -- no title, issue or series information beyond
+    Operator = auto()
+    Calendar = auto()
+    InfoSpecifier = auto()  # Specifies type of info e.g. v1 for 'volume': 1
+    ArchiveType = auto()
+    Honorific = auto()
+    Keywords = auto()
+    FCBD = auto()
+    ComicType = auto()
+    Publisher = auto()
+    C2C = auto()
+
+
+braces = [
+    ItemType.LeftBrace,
+    ItemType.LeftParen,
+    ItemType.LeftSBrace,
+    ItemType.RightBrace,
+    ItemType.RightParen,
+    ItemType.RightSBrace,
+]
+
+eof = chr(0)
+
+key = {
+    "fcbd": ItemType.FCBD,
+    "freecomicbookday": ItemType.FCBD,
+    "cbr": ItemType.ArchiveType,
+    "cbz": ItemType.ArchiveType,
+    "cbt": ItemType.ArchiveType,
+    "cb7": ItemType.ArchiveType,
+    "rar": ItemType.ArchiveType,
+    "zip": ItemType.ArchiveType,
+    "tar": ItemType.ArchiveType,
+    "7z": ItemType.ArchiveType,
+    "annual": ItemType.ComicType,
+    "book": ItemType.ComicType,
+    "volume": ItemType.InfoSpecifier,
+    "vol.": ItemType.InfoSpecifier,
+    "vol": ItemType.InfoSpecifier,
+    "v": ItemType.InfoSpecifier,
+    "of": ItemType.InfoSpecifier,
+    "dc": ItemType.Publisher,
+    "marvel": ItemType.Publisher,
+    "covers": ItemType.InfoSpecifier,
+    "c2c": ItemType.C2C,
+    "mr": ItemType.Honorific,
+    "ms": ItemType.Honorific,
+    "mrs": ItemType.Honorific,
+    "dr": ItemType.Honorific,
+}
+
+
+class Item:
+    def __init__(self, typ: ItemType, pos: int, val: str):
+        self.typ: ItemType = typ
+        self.pos: int = pos
+        self.val: str = val
+
+    def __repr__(self):
+        return f"{self.val}: index: {self.pos}: {self.typ}"
+
+
+class Lexer:
+    def __init__(self, string):
+        self.input: str = string  # The string being scanned
+        self.state = None  # The next lexing function to enter
+        self.pos: int = -1  # Current position in the input
+        self.start: int = 0  # Start position of this item
+        self.lastPos: int = 0  # Position of most recent item returned by nextItem
+        self.paren_depth: int = 0  # Nesting depth of ( ) exprs
+        self.brace_depth: int = 0  # Nesting depth of { }
+        self.sbrace_depth: int = 0  # Nesting depth of [ ]
+        self.items = []
+
+    # Next returns the next rune in the input.
+    def get(self) -> str:
+        if int(self.pos) >= len(self.input) - 1:
+            self.pos += 1
+            return eof
+
+        self.pos += 1
+        return self.input[self.pos]
+
+    # Peek returns but does not consume the next rune in the input.
+    def peek(self) -> str:
+        if int(self.pos) >= len(self.input) - 1:
+            return eof
+
+        return self.input[self.pos + 1]
+
+    def backup(self):
+        self.pos -= 1
+
+    # Emit passes an item back to the client.
+    def emit(self, t: ItemType):
+        self.items.append(Item(t, self.start, self.input[self.start : self.pos + 1]))
+        self.start = self.pos + 1
+
+    # Ignore skips over the pending input before this point.
+    def ignore(self):
+        self.start = self.pos
+
+    # Accept consumes the next rune if it's from the valid se:
+    def accept(self, valid: str):
+        if self.get() in valid:
+            return True
+
+        self.backup()
+        return False
+
+    # AcceptRun consumes a run of runes from the valid set.
+    def accept_run(self, valid: str):
+        while self.get() in valid:
+            pass
+
+        self.backup()
+
+    # Errorf returns an error token and terminates the scan by passing
+    # Back a nil pointer that will be the next state, terminating self.nextItem.
+    def errorf(self, message: str):
+        self.items.append(Item(ItemType.Error, self.start, message))
+
+    # NextItem returns the next item from the input.
+    # Called by the parser, not in the lexing goroutine.
+    # def next_item(self) -> Item:
+    #     item: Item = self.items.get()
+    #     self.lastPos = item.pos
+    #     return item
+
+    def scan_number(self):
+        digits = "0123456789"
+
+        self.accept_run(digits)
+        if self.accept("."):
+            if self.accept(digits):
+                self.accept_run(digits)
+            else:
+                self.backup()
+        if self.accept("s"):
+            if not self.accept("t"):
+                self.backup()
+        elif self.accept("nr"):
+            if not self.accept("d"):
+                self.backup()
+        elif self.accept("t"):
+            if not self.accept("h"):
+                self.backup()
+
+        return True
+
+    # Runs the state machine for the lexer.
+    def run(self):
+        self.state = lex_filename
+        while self.state is not None:
+            self.state = self.state(self)
+
+
+# Scans the elements inside action delimiters.
+def lex_filename(lex: Lexer):
+    r = lex.get()
+    if r == eof:
+        if lex.paren_depth != 0:
+            return lex.errorf("unclosed left paren")
+
+        if lex.brace_depth != 0:
+            return lex.errorf("unclosed left paren")
+        lex.emit(ItemType.EOF)
+        return None
+    elif is_space(r):
+        if r == "_" and lex.peek() == "_":
+            lex.get()
+            lex.emit(ItemType.Skip)
+        else:
+            return lex_space
+    elif r == ".":
+        r = lex.peek()
+        if r < "0" or "9" < r:
+            lex.emit(ItemType.Dot)
+            return lex_filename
+
+        lex.backup()
+        return lex_number
+    elif r == "'":
+        r = lex.peek()
+        if r in "0123456789":
+            return lex_number
+        lex.emit(ItemType.Text)  # TODO: Change to Text
+    elif "0" <= r <= "9":
+        lex.backup()
+        return lex_number
+    elif r == "#":
+        if "0" <= lex.peek() <= "9":
+            return lex_number
+        lex.emit(ItemType.Symbol)
+    elif is_operator(r):
+        if r == "-" and lex.peek() == "-":
+            lex.get()
+            lex.emit(ItemType.Skip)
+        else:
+            return lex_operator
+    elif is_alpha_numeric(r):
+        lex.backup()
+        return lex_text
+    elif r == "(":
+        lex.emit(ItemType.LeftParen)
+        lex.paren_depth += 1
+    elif r == ")":
+        lex.emit(ItemType.RightParen)
+        lex.paren_depth -= 1
+        if lex.paren_depth < 0:
+            return lex.errorf("unexpected right paren " + r)
+
+    elif r == "{":
+        lex.emit(ItemType.LeftBrace)
+        lex.brace_depth += 1
+    elif r == "}":
+        lex.emit(ItemType.RightBrace)
+        lex.brace_depth -= 1
+        if lex.brace_depth < 0:
+            return lex.errorf("unexpected right brace " + r)
+
+    elif r == "[":
+        lex.emit(ItemType.LeftSBrace)
+        lex.sbrace_depth += 1
+    elif r == "]":
+        lex.emit(ItemType.RightSBrace)
+        lex.sbrace_depth -= 1
+        if lex.sbrace_depth < 0:
+            return lex.errorf("unexpected right brace " + r)
+    elif is_symbol(r):
+        # L.backup()
+        lex.emit(ItemType.Symbol)
+    else:
+        return lex.errorf("unrecognized character in action: " + r)
+
+    return lex_filename
+
+
+def lex_operator(lex: Lexer):
+    lex.accept_run("-|:;")
+    lex.emit(ItemType.Operator)
+    return lex_filename
+
+
+# LexSpace scans a run of space characters.
+# One space has already been seen.
+def lex_space(lex: Lexer):
+    while is_space(lex.peek()):
+        lex.get()
+
+    lex.emit(ItemType.Space)
+    return lex_filename
+
+
+# Lex_text scans an alphanumeric.
+def lex_text(lex: Lexer):
+    while True:
+        r = lex.get()
+        if is_alpha_numeric(r):
+            if r.isnumeric():  # E.g. v1
+                word = lex.input[lex.start : lex.pos]
+                if word.lower() in key and key[word.lower()] == ItemType.InfoSpecifier:
+                    lex.backup()
+                    lex.emit(key[word.lower()])
+                    return lex_filename
+        else:
+            if r == "'" and lex.peek() == "s":
+                lex.get()
+            else:
+                lex.backup()
+            word = lex.input[lex.start : lex.pos + 1]
+            if word.lower() == "vol" and lex.peek() == ".":
+                lex.get()
+            word = lex.input[lex.start : lex.pos + 1]
+
+            if word.lower() in key:
+                lex.emit(key[word.lower()])
+            elif cal(word):
+                lex.emit(ItemType.Calendar)
+            else:
+                lex.emit(ItemType.Text)
+            break
+
+    return lex_filename
+
+
+def cal(value: str):
+    month_abbr = [i for i, x in enumerate(calendar.month_abbr) if x == value.title()]
+    month_name = [i for i, x in enumerate(calendar.month_name) if x == value.title()]
+    day_abbr = [i for i, x in enumerate(calendar.day_abbr) if x == value.title()]
+    day_name = [i for i, x in enumerate(calendar.day_name) if x == value.title()]
+    return set(month_abbr + month_name + day_abbr + day_name)
+
+
+def lex_number(lex: Lexer):
+    if not lex.scan_number():
+        return lex.errorf("bad number syntax: " + lex.input[lex.start : lex.pos])
+    # Complex number logic removed. Messes with math operations without space
+
+    if lex.input[lex.start] == "#":
+        lex.emit(ItemType.IssueNumber)
+    elif not lex.input[lex.pos].isdigit():
+        # Assume that 80th is just text and not a number
+        lex.emit(ItemType.Text)
+    else:
+        lex.emit(ItemType.Number)
+
+    return lex_filename
+
+
+def is_space(character: str):
+    return character in "_ \t"
+
+
+# IsAlphaNumeric reports whether r is an alphabetic, digit, or underscore.
+def is_alpha_numeric(character: str):
+    return character.isalpha() or character.isnumeric()
+
+
+def is_operator(character: str):
+    return character in "-|:;/\\"
+
+
+def is_symbol(character: str):
+    return unicodedata.category(character)[0] in "PS"
+
+
+def Lex(filename: str):
+    lex = Lexer(string=os.path.basename(filename))
+    lex.run()
+    return lex

comicapi/filenameparser.py

@@ -23,8 +23,17 @@ This should probably be re-written, but, well, it mostly works!
 import logging
 import os
 import re
+from operator import itemgetter
+from typing import TypedDict
 from urllib.parse import unquote
 
+from text2digits import text2digits
+
+from comicapi import filenamelexer, issuestring
+
+t2d = text2digits.Text2Digits(add_ordinal_ending=False)
+t2do = text2digits.Text2Digits(add_ordinal_ending=True)
+
 logger = logging.getLogger(__name__)
 
 
@@ -68,9 +77,7 @@ class FileNameParser:
             if match:
                 count = match.group()
 
-        count = count.lstrip("0")
-
-        return count
+        return count.lstrip("0")
 
     def get_issue_number(self, filename):
         """Returns a tuple of issue number string, and start and end indexes in the filename
@@ -89,7 +96,7 @@ class FileNameParser:
             # is the series name followed by issue
             filename = re.sub(r"--.*", self.repl, filename)
 
-        elif "__" in filename:
+        elif "__" in filename and not re.search(r"\[__\d+__]", filename):
             # the pattern seems to be that anything to left of the first "__"
             # is the series name followed by issue
             filename = re.sub(r"__.*", self.repl, filename)
@@ -107,8 +114,6 @@ class FileNameParser:
         # some titles)
         filename = re.sub(r"of [\d]+", self.repl, filename)
 
-        # print u"[{0}]".format(filename)
-
         # we should now have a cleaned up filename version with all the words in
         # the same positions as original filename
 
@@ -224,7 +229,7 @@ class FileNameParser:
 
         year = ""
         # look for four digit number with "(" ")" or "--" around it
-        match = re.search(r"(\(\d\d\d\d\))|(--\d\d\d\d--)", filename)
+        match = re.search(r"(\(\d{4}\))|(--\d{4}--)", filename)
         if match:
             year = match.group()
             # remove non-digits
@@ -292,3 +297,814 @@ class FileNameParser:
                 self.issue = "0"
             if self.issue[0] == ".":
                 self.issue = "0" + self.issue
+
+
+class FilenameInfo(TypedDict, total=False):
+    alternate: str
+    annual: bool
+    archive: str
+    c2c: bool
+    fcbd: bool
+    issue: str
+    issue_count: str
+    publisher: str
+    remainder: str
+    series: str
+    title: str
+    volume: str
+    volume_count: str
+    year: str
+
+
+eof = filenamelexer.Item(filenamelexer.ItemType.EOF, -1, "")
+
+
+class Parser:
+    """docstring for FilenameParser"""
+
+    def __init__(
+        self,
+        lexer_result: list[filenamelexer.Item],
+        first_is_alt=False,
+        remove_c2c=False,
+        remove_fcbd=False,
+        remove_publisher=False,
+    ):
+        self.state = None
+        self.pos = -1
+
+        self.firstItem = True
+        self.skip = False
+        self.alt = False
+        self.filename_info: FilenameInfo = {"series": ""}
+        self.issue_number_at = None
+        self.in_something = 0  # In some sort of brackets {}[]()
+        self.in_brace = 0  # In {}
+        self.in_s_brace = 0  # In []
+        self.in_paren = 0  # In ()
+        self.year_candidates: list[tuple[bool, filenamelexer.Item]] = []
+        self.series_parts: list[filenamelexer.Item] = []
+        self.title_parts: list[filenamelexer.Item] = []
+        self.used_items: list[filenamelexer.Item] = []
+        self.irrelevant: list[filenamelexer.Item] = []
+        self.operator_rejected: list[filenamelexer.Item] = []
+        self.publisher_removed: list[filenamelexer.Item] = []
+
+        self.first_is_alt = first_is_alt
+        self.remove_c2c = remove_c2c
+        self.remove_fcbd = remove_fcbd
+        self.remove_publisher = remove_publisher
+
+        self.input = lexer_result
+        for i, item in enumerate(self.input):
+            if item.typ == filenamelexer.ItemType.IssueNumber:
+                self.issue_number_at = i
+
+    # Get returns the next Item in the input.
+    def get(self) -> filenamelexer.Item:
+        if int(self.pos) >= len(self.input) - 1:
+            self.pos += 1
+            return eof
+
+        self.pos += 1
+        return self.input[self.pos]
+
+    # Peek returns but does not consume the next Item in the input.
+    def peek(self) -> filenamelexer.Item:
+        if int(self.pos) >= len(self.input) - 1:
+            return eof
+
+        return self.input[self.pos + 1]
+
+    # Peek_back returns but does not step back the previous Item in the input.
+    def peek_back(self) -> filenamelexer.Item:
+        if int(self.pos) == 0:
+            return eof
+
+        return self.input[self.pos - 1]
+
+    # Backup steps back one Item.
+    def backup(self):
+        self.pos -= 1
+
+    def run(self):
+        self.state = parse
+        while self.state is not None:
+            self.state = self.state(self)
+
+
+def parse(p: Parser):
+    item: filenamelexer.Item = p.get()
+
+    # We're done, time to do final processing
+    if item.typ == filenamelexer.ItemType.EOF:
+        return parse_finish
+
+    # Need to figure out if this is the issue number
+    if item.typ == filenamelexer.ItemType.Number:
+        likely_year = False
+        if p.firstItem and p.first_is_alt:
+            # raise Exception("fuck you")
+            p.alt = True
+            return parse_issue_number
+
+        # The issue number should hopefully not be in parentheses
+        if p.in_something == 0:
+            # Assume that operators indicate a non-issue number e.g. IG-88 or 88-IG
+            if filenamelexer.ItemType.Operator not in (p.peek().typ, p.peek_back().typ):
+                # It is common to use '89 to refer to an annual reprint from 1989
+                if item.val[0] != "'":
+                    # Issue number is less than 4 digits. very few series go above 999
+                    if len(item.val.lstrip("0")) < 4:
+                        # An issue number starting with # Was not found and no previous number was found
+                        if p.issue_number_at is None:
+                            # Series has already been started/parsed, filters out leading alternate numbers leading alternate number
+                            if len(p.series_parts) > 0:
+                                # Unset first item
+                                if p.firstItem:
+                                    p.firstItem = False
+                                return parse_issue_number
+            else:
+                p.operator_rejected.append(item)
+                # operator rejected used later to add back to the series/title
+
+        # It is more likely to be a year if it is inside parentheses.
+        if p.in_something > 0:
+            likely_year = True
+
+        # If numbers are directly followed by text it most likely isn't a year e.g. 2048px
+        if p.peek().typ == filenamelexer.ItemType.Text:
+            likely_year = False
+
+        # Is either a full year '2001' or a short year "'89"
+        if len(item.val) == 4 or item.val[0] == "'":
+            if p.in_something == 0:
+                # Append to series in case it is a part of the title, but only if were not inside parenthesis
+                p.series_parts.append(item)
+
+            # Look for a full date as in 2022-04-22
+            if p.peek().typ in [
+                filenamelexer.ItemType.Symbol,
+                filenamelexer.ItemType.Operator,
+                filenamelexer.ItemType.Dot,
+            ]:
+                op = [p.get()]
+                if p.peek().typ == filenamelexer.ItemType.Number:
+                    month = p.get()
+                    if p.peek().typ in [
+                        filenamelexer.ItemType.Symbol,
+                        filenamelexer.ItemType.Operator,
+                        filenamelexer.ItemType.Dot,
+                    ]:
+                        op.append(p.get())
+                        if p.peek().typ == filenamelexer.ItemType.Number:
+                            day = p.get()
+                            fulldate = [month, day, item]
+                            p.used_items.extend(op)
+                            p.used_items.extend(fulldate)
+                        else:
+                            p.backup()
+                            p.backup()
+                            p.backup()
+                            # TODO never happens
+                    else:
+                        p.backup()
+                        p.backup()
+                        # TODO never happens
+                else:
+                    p.backup()
+                    # TODO never happens
+
+            p.year_candidates.append((likely_year, item))
+        # Ensures that IG-88 gets added back to the series/title
+        elif (
+            p.in_something == 0
+            and p.peek_back().typ == filenamelexer.ItemType.Operator
+            or p.peek().typ == filenamelexer.ItemType.Operator
+        ):
+            # Were not in something and the next or previous type is an operator, add it to the series
+            p.series_parts.append(item)
+            p.used_items.append(item)
+
+            # Unset first item
+            if p.firstItem:
+                p.firstItem = False
+            p.get()
+            return parse_series
+
+    # Number with a leading hash e.g. #003
+    elif item.typ == filenamelexer.ItemType.IssueNumber:
+        # Unset first item
+        if p.firstItem:
+            p.firstItem = False
+        return parse_issue_number
+
+    # Matches FCBD. Not added to p.used_items so it will show in "remainder"
+    elif item.typ == filenamelexer.ItemType.FCBD:
+        p.filename_info["fcbd"] = True
+
+    # Matches c2c. Not added to p.used_items so it will show in "remainder"
+    elif item.typ == filenamelexer.ItemType.C2C:
+        p.filename_info["c2c"] = True
+
+    # Matches the extension if it is known to be an archive format e.g. cbt,cbz,zip,rar
+    elif item.typ == filenamelexer.ItemType.ArchiveType:
+        p.filename_info["archive"] = item.val.lower()
+        p.used_items.append(item)
+        if p.peek_back().typ == filenamelexer.ItemType.Dot:
+            p.used_items.append(p.peek_back())
+
+    # Allows removing DC from 'Wonder Woman 49 DC Sep-Oct 1951' dependent on publisher being in a static list in the lexer
+    elif item.typ == filenamelexer.ItemType.Publisher:
+        p.filename_info["publisher"] = item.val
+        p.used_items.append(item)
+        if p.firstItem:
+            p.firstItem = False
+            if p.in_something == 0:
+                return parse_series
+        p.publisher_removed.append(item)
+        if p.in_something == 0:
+            return parse_series
+
+    # Attempts to identify the type e.g. annual
+    elif item.typ == filenamelexer.ItemType.ComicType:
+        series_append = True
+
+        if p.peek().typ == filenamelexer.ItemType.Space:
+            p.get()
+
+        if p.series_parts and "free comic book" in (" ".join([x.val for x in p.series_parts]) + " " + item.val).lower():
+            p.filename_info["fcbd"] = True
+            series_append = True
+        # If the next item is a number it's probably the volume
+        elif p.peek().typ == filenamelexer.ItemType.Number or (
+            p.peek().typ == filenamelexer.ItemType.Text and t2d.convert(p.peek().val).isnumeric()
+        ):
+            number = p.get()
+            # Mark volume info. Text will be added to the title/series later
+            if item.val.lower() in ["book", "tpb"]:
+                p.title_parts.extend([item, number])
+                p.filename_info["volume"] = t2do.convert(number.val)
+                p.filename_info["issue"] = t2do.convert(number.val)
+
+                p.used_items.append(item)
+                series_append = False
+
+            # Annuals usually mean the year
+            elif item.val.lower() in ["annual"]:
+                p.filename_info["annual"] = True
+                num = t2d.convert(number.val)
+                if num.isnumeric() and len(num) == 4:
+                    p.year_candidates.append((True, number))
+                else:
+                    p.backup()
+
+        elif item.val.lower() in ["annual"]:
+            p.filename_info["annual"] = True
+
+        # If we don't have a reason to exclude it from the series go back to parsing the series immediately
+        if series_append:
+            p.series_parts.append(item)
+            p.used_items.append(item)
+            return parse_series
+
+    # We found text, it's probably the title or series
+    elif item.typ in [filenamelexer.ItemType.Text, filenamelexer.ItemType.Honorific]:
+        # Unset first item
+        if p.firstItem:
+            p.firstItem = False
+        if p.in_something == 0:
+            return parse_series
+
+    # Usually the word 'of' eg 1 (of 6)
+    elif item.typ == filenamelexer.ItemType.InfoSpecifier:
+        return parse_info_specifier
+
+    # Operator is a symbol that acts as some sort of separator eg - : ;
+    elif item.typ == filenamelexer.ItemType.Operator:
+        if p.in_something == 0:
+            p.irrelevant.append(item)
+
+    # Filter out Month and day names in filename
+    elif item.typ == filenamelexer.ItemType.Calendar:
+        # Month and day are currently irrelevant if they are inside parentheses e.g. (January 2002)
+        if p.in_something > 0:
+            p.irrelevant.append(item)
+
+        # assume Sep-Oct is not useful in the series/title
+        elif p.peek().typ in [filenamelexer.ItemType.Symbol, filenamelexer.ItemType.Operator]:
+            p.get()
+            if p.peek().typ == filenamelexer.ItemType.Calendar:
+                p.irrelevant.extend([item, p.input[p.pos], p.get()])
+            else:
+                p.backup()
+                return parse_series
+        # This is text that just happens to also be a month/day
+        else:
+            return parse_series
+
+    # Specifically '__' or '--', no further title/series parsing is done to keep compatibility with wiki
+    elif item.typ == filenamelexer.ItemType.Skip:
+        p.skip = True
+
+    # Keeping track of parentheses depth
+    elif item.typ == filenamelexer.ItemType.LeftParen:
+        p.in_paren += 1
+        p.in_something += 1
+    elif item.typ == filenamelexer.ItemType.LeftBrace:
+        p.in_brace += 1
+        p.in_something += 1
+    elif item.typ == filenamelexer.ItemType.LeftSBrace:
+        p.in_s_brace += 1
+        p.in_something += 1
+
+    elif item.typ == filenamelexer.ItemType.RightParen:
+        p.in_paren -= 1
+        p.in_something -= 1
+    elif item.typ == filenamelexer.ItemType.RightBrace:
+        p.in_brace -= 1
+        p.in_something -= 1
+    elif item.typ == filenamelexer.ItemType.RightSBrace:
+        p.in_s_brace -= 1
+        p.in_something -= 1
+
+    # Unset first item
+    if p.firstItem:
+        p.firstItem = False
+
+    # Brace management, I don't like negative numbers
+    if p.in_paren < 0:
+        p.in_something += p.in_paren * -1
+    if p.in_brace < 0:
+        p.in_something += p.in_brace * -1
+    if p.in_s_brace < 0:
+        p.in_something += p.in_s_brace * -1
+
+    return parse
+
+
+# TODO: What about more esoteric numbers???
+def parse_issue_number(p: Parser):
+    item = p.input[p.pos]
+
+    if "issue" in p.filename_info:
+        if "alternate" in p.filename_info:
+            p.filename_info["alternate"] += "," + item.val
+        p.filename_info["alternate"] = item.val
+    else:
+        if p.alt:
+            p.filename_info["alternate"] = item.val
+        else:
+            p.filename_info["issue"] = item.val
+            p.issue_number_at = item.pos
+        p.used_items.append(item)
+        item = p.get()
+        if item.typ == filenamelexer.ItemType.Dot:
+            p.used_items.append(item)
+            item = p.get()
+            if item.typ in [filenamelexer.ItemType.Text, filenamelexer.ItemType.Number]:
+                if p.alt:
+                    p.filename_info["alternate"] += "." + item.val
+                else:
+                    p.filename_info["issue"] += "." + item.val
+                p.used_items.append(item)
+            else:
+                p.backup()
+                p.backup()
+        else:
+            p.backup()
+    p.alt = False
+    return parse
+
+
+def parse_series(p: Parser):
+    item = p.input[p.pos]
+
+    series: list[list[filenamelexer.Item]] = [[]]
+    # Space and Dots are not useful at the beginning of a title/series
+    if not p.skip and item.typ not in [filenamelexer.ItemType.Space, filenamelexer.ItemType.Dot]:
+        series[0].append(item)
+
+    current_part = 0
+
+    title_parts: list[filenamelexer.Item] = []
+    series_parts: list[filenamelexer.Item] = []
+
+    prev_space = False
+
+    # 'free comic book day' screws things up. #TODO look into removing book from ComicType?
+
+    # We stop parsing the series when certain things come up if nothing was done with them continue where we left off
+    if (
+        p.series_parts
+        and p.series_parts[-1].val.lower() == "book"
+        or p.peek_back().typ == filenamelexer.ItemType.Number
+        or item.typ == filenamelexer.ItemType.Calendar
+    ):
+        series_parts = p.series_parts
+        p.series_parts = []
+    # Skip is only true if we have come across '--' or '__'
+    while not p.skip:
+        item = p.get()
+
+        # Spaces are evil
+        if item.typ == filenamelexer.ItemType.Space:
+            prev_space = True
+            continue
+        if item.typ in [
+            filenamelexer.ItemType.Text,
+            filenamelexer.ItemType.Symbol,
+            filenamelexer.ItemType.Publisher,
+            filenamelexer.ItemType.Honorific,
+        ]:
+            series[current_part].append(item)
+            if item.typ == filenamelexer.ItemType.Honorific and p.peek().typ == filenamelexer.ItemType.Dot:
+                series[current_part].append(p.get())
+            elif item.typ == filenamelexer.ItemType.Publisher:
+                p.filename_info["publisher"] = item.val
+
+        # Handle Volume
+        elif item.typ == filenamelexer.ItemType.InfoSpecifier:
+            # Exception for 'of'
+            if item.val.lower() == "of":
+                series[current_part].append(item)
+            else:
+                # This specifically lets 'X-Men-V1-067' parse correctly as Series: X-Men Volume: 1 Issue: 67
+                while len(series[current_part]) > 0 and series[current_part][-1].typ not in [
+                    filenamelexer.ItemType.Text,
+                    filenamelexer.ItemType.Symbol,
+                ]:
+                    p.irrelevant.append(series[current_part].pop())
+                p.backup()
+                break
+
+        elif item.typ == filenamelexer.ItemType.Operator:
+            peek = p.peek()
+            # ': ' separates the title from the series, only the last section is considered the title
+            if not prev_space and peek.typ in [filenamelexer.ItemType.Space]:
+                series.append([])  # Starts a new section
+                series[current_part].append(item)
+                current_part += 1
+            else:
+                # Force space around '-' makes 'batman - superman' stay otherwise we get 'batman-superman'
+                if prev_space and peek.typ in [filenamelexer.ItemType.Space]:
+                    item.val = " " + item.val + " "
+                series[current_part].append(item)
+
+        # Stop processing series/title if a skip item is found
+        elif item.typ == filenamelexer.ItemType.Skip:
+            p.backup()
+            break
+
+        elif item.typ == filenamelexer.ItemType.Number:
+            if p.peek().typ == filenamelexer.ItemType.Space:
+                p.get()
+                # We have 2 numbers, add the first to the series and then go back to parse
+                if p.peek().typ == filenamelexer.ItemType.Number:
+                    series[current_part].append(item)
+                    break
+
+                # We have 1 number break here, it's possible it's the issue
+                p.backup()  # Whitespace
+                p.backup()  # The number
+                break
+            # This is 6 in '1 of 6'
+            if series[current_part] and series[current_part][-1].val.lower() == "of":
+                series[current_part].append(item)
+
+            # We have 1 number break here, it's possible it's the issue
+            else:
+                p.backup()  # The number
+                break
+
+        else:
+            # Ensure 'ms. marvel' parses 'ms.' correctly
+            if item.typ == filenamelexer.ItemType.Dot and p.peek_back().typ == filenamelexer.ItemType.Honorific:
+                series[current_part].append(item)
+            # Allows avengers.hulk to parse correctly
+            elif item.typ == filenamelexer.ItemType.Dot and p.peek().typ == filenamelexer.ItemType.Text:
+                # Marks the dot as used so that the remainder is clean
+                p.used_items.append(item)
+            else:
+                p.backup()
+                break
+
+        prev_space = False
+
+    # We have a title separator e.g. ': "
+    if len(series) > 1:
+        title_parts.extend(series.pop())
+        for s in series:
+            if s and s[-1].typ == filenamelexer.ItemType.Operator:
+                s[-1].val += " "  # Ensures that when there are multiple separators that they display properly
+            series_parts.extend(s)
+        p.used_items.append(series_parts.pop())
+    else:
+        series_parts.extend(series[0])
+
+    # If the series has already been set assume all of this is the title.
+    if len(p.series_parts) > 0:
+        p.title_parts.extend(series_parts)
+        p.title_parts.extend(title_parts)
+    else:
+        p.series_parts.extend(series_parts)
+        p.title_parts.extend(title_parts)
+    return parse
+
+
+def resolve_year(p: Parser):
+    if len(p.year_candidates) > 0:
+        # Sort by likely_year boolean
+        p.year_candidates.sort(key=itemgetter(0))
+
+        # Take the last year e.g. (2007) 2099 (2008) becomes 2099 2007 2008 and takes 2008
+        selected_year = p.year_candidates.pop()[1]
+
+        p.filename_info["year"] = selected_year.val
+        p.used_items.append(selected_year)
+
+        # (2008) Title (2009) is many times used to denote the series year if we don't have a volume we use it
+        if "volume" not in p.filename_info and p.year_candidates and p.year_candidates[-1][0]:
+            vol = p.year_candidates.pop()[1]
+            p.filename_info["volume"] = vol.val
+            p.used_items.append(vol)
+
+            # Remove volume from series and title
+            if selected_year in p.series_parts:
+                p.series_parts.remove(selected_year)
+            if selected_year in p.title_parts:
+                p.title_parts.remove(selected_year)
+
+        # Remove year from series and title
+        if selected_year in p.series_parts:
+            p.series_parts.remove(selected_year)
+        if selected_year in p.title_parts:
+            p.title_parts.remove(selected_year)
+
+
+def parse_finish(p: Parser):
+    resolve_year(p)
+
+    # If we don't have an issue try to find it in the series
+    if "issue" not in p.filename_info and p.series_parts and p.series_parts[-1].typ == filenamelexer.ItemType.Number:
+        issue_num = p.series_parts.pop()
+
+        # If the number we just popped is a year put it back on it's probably part of the series e.g. Spider-Man 2099
+        if issue_num in [x[1] for x in p.year_candidates]:
+            p.series_parts.append(issue_num)
+        else:
+            # If this number was rejected because of an operator and the operator is still there add it back e.g. 'IG-88'
+            if (
+                issue_num in p.operator_rejected
+                and p.series_parts
+                and p.series_parts[-1].typ == filenamelexer.ItemType.Operator
+            ):
+                p.series_parts.append(issue_num)
+            # We have no reason to not use this number as the issue number. Specifically happens when parsing 'X-Men-V1-067.cbr'
+            else:
+                p.filename_info["issue"] = issue_num.val
+                p.used_items.append(issue_num)
+                p.issue_number_at = issue_num.pos
+
+    # Remove publishers, currently only marvel and dc are defined,
+    # this is an option specifically because this can drastically screw up parsing
+    if p.remove_publisher:
+        for item in p.publisher_removed:
+            if item in p.series_parts:
+                p.series_parts.remove(item)
+            if item in p.title_parts:
+                p.title_parts.remove(item)
+
+    p.filename_info["series"] = join_title(p.series_parts)
+    p.used_items.extend(p.series_parts)
+
+    p.filename_info["title"] = join_title(p.title_parts)
+    p.used_items.extend(p.title_parts)
+
+    if "issue" in p.filename_info:
+        p.filename_info["issue"] = issuestring.IssueString(p.filename_info["issue"].lstrip("#")).as_string()
+
+    if "volume" in p.filename_info:
+        p.filename_info["volume"] = p.filename_info["volume"].lstrip("#").lstrip("0")
+
+    if "issue" not in p.filename_info:
+        # We have an alternate move it to the issue
+        if "alternate" in p.filename_info:
+            p.filename_info["issue"] = p.filename_info["alternate"]
+            p.filename_info["alternate"] = ""
+        else:
+            # TODO: This never happens
+            inp = [x for x in p.input if x not in p.irrelevant and x not in p.used_items and x.typ != eof.typ]
+            if len(inp) == 1 and inp[0].typ == filenamelexer.ItemType.Number:
+                p.filename_info["issue"] = inp[0].val
+                p.used_items.append(inp[0])
+
+    remove_items = []
+    if p.remove_fcbd:
+        remove_items.append(filenamelexer.ItemType.FCBD)
+    if p.remove_c2c:
+        remove_items.append(filenamelexer.ItemType.C2C)
+
+    p.irrelevant.extend([x for x in p.input if x.typ in remove_items])
+
+    p.filename_info["remainder"] = get_remainder(p)
+
+    # Ensure keys always exist
+    for s in [
+        "alternate",
+        "issue",
+        "archive",
+        "series",
+        "title",
+        "volume",
+        "year",
+        "remainder",
+        "issue_count",
+        "volume_count",
+        "publisher",
+    ]:
+        if s not in p.filename_info:
+            p.filename_info[s] = ""
+    for s in ["fcbd", "c2c", "annual"]:
+        if s not in p.filename_info:
+            p.filename_info[s] = False
+
+
+def get_remainder(p: Parser):
+    remainder = ""
+    rem = []
+
+    # Remove used items and irrelevant items e.g. the series and useless operators
+    inp = [x for x in p.input if x not in p.irrelevant and x not in p.used_items]
+    for i, item in enumerate(inp):
+        # No double space or space next to parentheses
+        if item.typ in [filenamelexer.ItemType.Space, filenamelexer.ItemType.Skip]:
+            if (
+                i > 0
+                and inp[i - 1].typ
+                not in [
+                    filenamelexer.ItemType.Space,
+                    filenamelexer.ItemType.LeftBrace,
+                    filenamelexer.ItemType.LeftParen,
+                    filenamelexer.ItemType.LeftSBrace,
+                ]
+                and i + 1 < len(inp)
+                and inp[i + 1].typ
+                not in [
+                    filenamelexer.ItemType.RightBrace,
+                    filenamelexer.ItemType.RightParen,
+                    filenamelexer.ItemType.RightSBrace,
+                ]
+            ):
+                remainder += " "
+
+        # Strip off useless opening parenthesis
+        elif (
+            item.typ
+            in [
+                filenamelexer.ItemType.Space,
+                filenamelexer.ItemType.RightBrace,
+                filenamelexer.ItemType.RightParen,
+                filenamelexer.ItemType.RightSBrace,
+            ]
+            and i > 0
+            and inp[i - 1].typ
+            in [
+                filenamelexer.ItemType.LeftBrace,
+                filenamelexer.ItemType.LeftParen,
+                filenamelexer.ItemType.LeftSBrace,
+            ]
+        ):
+            remainder = remainder.rstrip("[{(")
+            continue
+
+        # Add the next item
+        else:
+            rem.append(item)
+            remainder += item.val
+
+    # Remove empty parentheses
+    remainder = re.sub(r"[\[{(]+[]})]+", "", remainder)
+    return remainder.strip()
+
+
+def parse_info_specifier(p: Parser):
+    item = p.input[p.pos]
+    index = p.pos
+
+    if p.peek().typ == filenamelexer.ItemType.Space:
+        p.get()
+
+    # Handles 'book 3' and 'book three'
+    if p.peek().typ == filenamelexer.ItemType.Number or (
+        p.peek().typ == filenamelexer.ItemType.Text and t2d.convert(p.peek().val).isnumeric()
+    ):
+
+        number = p.get()
+        if item.val.lower() in ["volume", "vol", "vol.", "v"]:
+            p.filename_info["volume"] = t2do.convert(number.val)
+            p.used_items.append(item)
+            p.used_items.append(number)
+
+        # 'of' is only special if it is inside a parenthesis.
+        elif item.val.lower() == "of":
+            i = get_number(p, index)
+            if p.in_something > 0:
+                if p.issue_number_at is None:
+                    # TODO: Figure out what to do here if it ever happens
+                    p.filename_info["issue_count"] = str(int(t2do.convert(number.val)))
+                    p.used_items.append(item)
+                    p.used_items.append(number)
+
+                # This is definitely the issue number
+                elif p.issue_number_at == i.pos:
+                    p.filename_info["issue_count"] = str(int(t2do.convert(number.val)))
+                    p.used_items.append(item)
+                    p.used_items.append(number)
+
+                # This is not for the issue number it is not in either the issue or the title, assume it is the volume number and count
+                elif p.issue_number_at != i.pos and i not in p.series_parts and i not in p.title_parts:
+                    p.filename_info["volume"] = i.val
+                    p.filename_info["volume_count"] = str(int(t2do.convert(number.val)))
+                    p.used_items.append(i)
+                    p.used_items.append(item)
+                    p.used_items.append(number)
+                else:
+                    # TODO: Figure out what to do here if it ever happens
+                    pass
+            else:
+                # Lets 'The Wrath of Foobar-Man, Part 1 of 2' parse correctly as the title
+                if i is not None:
+                    p.pos = [ind for ind, x in enumerate(p.input) if x == i][0]
+
+            if not p.in_something:
+                return parse_series
+    return parse
+
+
+# Gets 03 in '03 of 6'
+def get_number(p: Parser, index: int):
+    # Go backward through the filename to see if we can find what this is of eg '03 (of 6)' or '008 title 03 (of 6)'
+    rev = p.input[:index]
+    rev.reverse()
+    for i in rev:
+        # We don't care about these types, we are looking to see if there is a number that is possibly different from the issue number for this count
+        if i.typ in [
+            filenamelexer.ItemType.LeftParen,
+            filenamelexer.ItemType.LeftBrace,
+            filenamelexer.ItemType.LeftSBrace,
+            filenamelexer.ItemType.Space,
+        ]:
+            continue
+        if i.typ == filenamelexer.ItemType.Number:
+            # We got our number, time to leave
+            return i
+        # This is not a number and not an ignorable type, give up looking for the number this count belongs to
+
+    return None
+
+
+def join_title(lst: list[filenamelexer.Item]):
+    title = ""
+    for i, item in enumerate(lst):
+        if i + 1 == len(lst) and item.val == ",":  # We ignore commas on the end
+            continue
+        title += item.val  # Add the next item
+        # No space after operators
+        if item.typ == filenamelexer.ItemType.Operator:
+            continue
+        # No trailing space
+        if i == len(lst) - 1:
+            continue
+        # No space after honorifics with a dot
+        if item.typ == filenamelexer.ItemType.Honorific and lst[i + 1].typ == filenamelexer.ItemType.Dot:
+            continue
+        # No space if the next item is an operator or symbol
+        if lst[i + 1].typ in [
+            filenamelexer.ItemType.Operator,
+            filenamelexer.ItemType.Symbol,
+        ]:
+            continue
+
+        # Add a space
+        title += " "
+
+    return title
+
+
+def Parse(
+    lexer_result: list[filenamelexer.Item],
+    first_is_alt=False,
+    remove_c2c=False,
+    remove_fcbd=False,
+    remove_publisher=False,
+):
+    p = Parser(
+        lexer_result=lexer_result,
+        first_is_alt=first_is_alt,
+        remove_c2c=remove_c2c,
+        remove_fcbd=remove_fcbd,
+        remove_publisher=remove_publisher,
+    )
+    p.run()
+    return p

comicapi/genericmetadata.py

@@ -48,8 +48,10 @@ class PageType:
     Deleted = "Deleted"
 
 
-class ImageMetadata(TypedDict):
-    Type: PageType
+class ImageMetadata(TypedDict, total=False):
+    Type: str
+    Bookmark: str
+    DoublePage: bool
     Image: int
     ImageSize: str
     ImageHeight: str
@@ -104,6 +106,7 @@ class GenericMetadata:
         self.black_and_white = None
         self.page_count = None
         self.maturity_rating = None
+        self.community_rating = None
 
         self.story_arc = None
         self.series_group = None
@@ -166,6 +169,7 @@ class GenericMetadata:
         assign("manga", new_md.manga)
         assign("black_and_white", new_md.black_and_white)
         assign("maturity_rating", new_md.maturity_rating)
+        assign("community_rating", new_md.community_rating)
         assign("story_arc", new_md.story_arc)
         assign("series_group", new_md.series_group)
         assign("scan_info", new_md.scan_info)
@@ -211,8 +215,7 @@ class GenericMetadata:
     def set_default_page_list(self, count):
         # generate a default page list, with the first page marked as the cover
         for i in range(count):
-            page_dict = {}
-            page_dict["Image"] = str(i)
+            page_dict = ImageMetadata(Image=i)
             if i == 0:
                 page_dict["Type"] = PageType.FrontCover
             self.pages.append(page_dict)
@@ -239,11 +242,7 @@ class GenericMetadata:
 
     def add_credit(self, person, role, primary=False):
 
-        credit = {}
-        credit["person"] = person
-        credit["role"] = role
-        if primary:
-            credit["primary"] = primary
+        credit: CreditMetadata = {"person": person, "role": role, "primary": primary}
 
         # look to see if it's not already there...
         found = False
@@ -257,6 +256,15 @@ class GenericMetadata:
         if not found:
             self.credits.append(credit)
 
+    def get_primary_credit(self, role):
+        primary = ""
+        for credit in self.credits:
+            if (primary == "" and credit["role"].lower() == role.lower()) or (
+                credit["role"].lower() == role.lower() and credit["primary"]
+            ):
+                primary = credit["person"]
+        return primary
+
     def __str__(self):
         vals = []
         if self.is_empty:
@@ -300,6 +308,7 @@ class GenericMetadata:
         if self.black_and_white:
             add_attr_string("black_and_white")
         add_attr_string("maturity_rating")
+        add_attr_string("community_rating")
         add_attr_string("story_arc")
         add_attr_string("series_group")
         add_attr_string("scan_info")
@@ -330,3 +339,94 @@ class GenericMetadata:
             outstr += fmt_str.format(i[0] + ":", i[1])
 
         return outstr
+
+
+md_test = GenericMetadata()
+
+md_test.is_empty = False
+md_test.tag_origin = None
+md_test.series = "Cory Doctorow's Futuristic Tales of the Here and Now"
+md_test.issue = "1"
+md_test.title = "Anda's Game"
+md_test.publisher = "IDW Publishing"
+md_test.month = 10
+md_test.year = 2007
+md_test.day = 1
+md_test.issue_count = 6
+md_test.volume = 1
+md_test.genre = "Sci-Fi"
+md_test.language = "en"
+md_test.comments = (
+    "For 12-year-old Anda, getting paid real money to kill the characters of players who were cheating in her favorite online "
+    "computer game was a win-win situation. Until she found out who was paying her, and what those characters meant to the "
+    "livelihood of children around the world."
+)
+md_test.volume_count = None
+md_test.critical_rating = None
+md_test.country = None
+md_test.alternate_series = "Tales"
+md_test.alternate_number = "2"
+md_test.alternate_count = 7
+md_test.imprint = "craphound.com"
+md_test.notes = "Tagged with ComicTagger 1.3.2a5 using info from Comic Vine on 2022-04-16 15:52:26. [Issue ID 140529]"
+md_test.web_link = "https://comicvine.gamespot.com/cory-doctorows-futuristic-tales-of-the-here-and-no/4000-140529/"
+md_test.format = "Series"
+md_test.manga = "No"
+md_test.black_and_white = None
+md_test.page_count = 24
+md_test.maturity_rating = "Everyone 10+"
+md_test.community_rating = "3.0"
+md_test.story_arc = "Here and Now"
+md_test.series_group = "Futuristic Tales"
+md_test.scan_info = "(CC BY-NC-SA 3.0)"
+md_test.characters = "Anda"
+md_test.teams = "Fahrenheit"
+md_test.locations = "lonely  cottage "
+md_test.credits = [
+    {"person": "Dara Naraghi", "role": "Writer"},
+    {"person": "Esteve Polls", "role": "Penciller"},
+    {"person": "Esteve Polls", "role": "Inker"},
+    {"person": "Neil Uyetake", "role": "Letterer"},
+    {"person": "Sam Kieth", "role": "Cover"},
+    {"person": "Ted Adams", "role": "Editor"},
+]
+md_test.tags = []
+md_test.pages = [
+    {"Image": 0, "ImageHeight": "1280", "ImageSize": "195977", "ImageWidth": "800", "Type": PageType.FrontCover},
+    {"Image": 1, "ImageHeight": "2039", "ImageSize": "611993", "ImageWidth": "1327"},
+    {"Image": 2, "ImageHeight": "2039", "ImageSize": "783726", "ImageWidth": "1327"},
+    {"Image": 3, "ImageHeight": "2039", "ImageSize": "679584", "ImageWidth": "1327"},
+    {"Image": 4, "ImageHeight": "2039", "ImageSize": "788179", "ImageWidth": "1327"},
+    {"Image": 5, "ImageHeight": "2039", "ImageSize": "864433", "ImageWidth": "1327"},
+    {"Image": 6, "ImageHeight": "2039", "ImageSize": "765606", "ImageWidth": "1327"},
+    {"Image": 7, "ImageHeight": "2039", "ImageSize": "876427", "ImageWidth": "1327"},
+    {"Image": 8, "ImageHeight": "2039", "ImageSize": "852622", "ImageWidth": "1327"},
+    {"Image": 9, "ImageHeight": "2039", "ImageSize": "800205", "ImageWidth": "1327"},
+    {"Image": 10, "ImageHeight": "2039", "ImageSize": "746243", "ImageWidth": "1326"},
+    {"Image": 11, "ImageHeight": "2039", "ImageSize": "718062", "ImageWidth": "1327"},
+    {"Image": 12, "ImageHeight": "2039", "ImageSize": "532179", "ImageWidth": "1326"},
+    {"Image": 13, "ImageHeight": "2039", "ImageSize": "686708", "ImageWidth": "1327"},
+    {"Image": 14, "ImageHeight": "2039", "ImageSize": "641907", "ImageWidth": "1327"},
+    {"Image": 15, "ImageHeight": "2039", "ImageSize": "805388", "ImageWidth": "1327"},
+    {"Image": 16, "ImageHeight": "2039", "ImageSize": "668927", "ImageWidth": "1326"},
+    {"Image": 17, "ImageHeight": "2039", "ImageSize": "710605", "ImageWidth": "1327"},
+    {"Image": 18, "ImageHeight": "2039", "ImageSize": "761398", "ImageWidth": "1326"},
+    {"Image": 19, "ImageHeight": "2039", "ImageSize": "743807", "ImageWidth": "1327"},
+    {"Image": 20, "ImageHeight": "2039", "ImageSize": "552911", "ImageWidth": "1326"},
+    {"Image": 21, "ImageHeight": "2039", "ImageSize": "556827", "ImageWidth": "1327"},
+    {"Image": 22, "ImageHeight": "2039", "ImageSize": "675078", "ImageWidth": "1326"},
+    {
+        "Bookmark": "Interview",
+        "Image": "23",
+        "ImageHeight": "2032",
+        "ImageSize": "800965",
+        "ImageWidth": "1338",
+        "Type": PageType.Letters,
+    },
+]
+md_test.price = None
+md_test.is_version_of = None
+md_test.rights = None
+md_test.identifier = None
+md_test.last_mark = None
+md_test.cover_image = None

comicapi/utils.py

@@ -33,23 +33,6 @@ class UtilsVars:
     already_fixed_encoding = False
 
 
-def indent(elem, level=0):
-    # for making the XML output readable
-    i = "\n" + level * "  "
-    if len(elem):
-        if not elem.text or not elem.text.strip():
-            elem.text = i + "  "
-        if not elem.tail or not elem.tail.strip():
-            elem.tail = i
-        for ele in elem:
-            indent(ele, level + 1)
-        if not elem.tail or not elem.tail.strip():
-            elem.tail = i
-    else:
-        if level and (not elem.tail or not elem.tail.strip()):
-            elem.tail = i
-
-
 def get_actual_preferred_encoding():
     preferred_encoding = locale.getpreferredencoding()
     if platform.system() == "Darwin":

comictaggerlib/autotagmatchwindow.py

@@ -32,11 +32,13 @@ logger = logging.getLogger(__name__)
 class AutoTagMatchWindow(QtWidgets.QDialog):
     volume_id = 0
 
-    def __init__(self, parent, match_set_list: List[MultipleMatch], style, fetch_func):
+    def __init__(self, parent, match_set_list: List[MultipleMatch], style, fetch_func, settings):
         super().__init__(parent)
 
         uic.loadUi(ComicTaggerSettings.get_ui_file("matchselectionwindow.ui"), self)
 
+        self.settings = settings
+
         self.current_match_set: Optional[MultipleMatch] = None
 
         self.altCoverWidget = CoverImageWidget(self.altCoverContainer, CoverImageWidget.AltCoverMode)
@@ -91,7 +93,7 @@ class AutoTagMatchWindow(QtWidgets.QDialog):
 
         path = self.current_match_set.ca.path
         self.setWindowTitle(
-            "Select correct match or skip ({0} of {1}): {2}".format(
+            "Select correct match or skip ({} of {}): {}".format(
                 self.current_match_set_idx + 1,
                 len(self.match_set_list),
                 os.path.split(path)[1],
@@ -221,7 +223,12 @@ class AutoTagMatchWindow(QtWidgets.QDialog):
 
         md = ca.read_metadata(self.style)
         if md.is_empty:
-            md = ca.metadata_from_filename()
+            md = ca.metadata_from_filename(
+                self.settings.complicated_parser,
+                self.settings.remove_c2c,
+                self.settings.remove_fcbd,
+                self.settings.remove_publisher,
+            )
 
         # now get the particular issue data
         cv_md = self.fetch_func(match)

comictaggerlib/cli.py

@@ -81,7 +81,7 @@ def display_match_set_for_choice(label, match_set: MultipleMatch, opts, settings
     for (counter, m) in enumerate(match_set.matches):
         counter += 1
         print(
-            "    {0}. {1} #{2} [{3}] ({4}/{5}) - {6}".format(
+            "    {}. {} #{} [{}] ({}/{}) - {}".format(
                 counter,
                 m["series"],
                 m["issue_number"],
@@ -101,7 +101,7 @@ def display_match_set_for_choice(label, match_set: MultipleMatch, opts, settings
             # save the data!
             # we know at this point, that the file is all good to go
             ca = match_set.ca
-            md = create_local_metadata(opts, ca, ca.has_metadata(opts.data_style))
+            md = create_local_metadata(opts, ca, ca.has_metadata(opts.data_style), settings)
             cv_md = actual_issue_data_fetch(match_set.matches[int(i)], settings, opts)
             md.overlay(cv_md)
             actual_metadata_save(ca, opts, md)
@@ -164,13 +164,17 @@ def cli_mode(opts, settings):
     post_process_matches(match_results, opts, settings)
 
 
-def create_local_metadata(opts, ca: ComicArchive, has_desired_tags):
+def create_local_metadata(opts, ca: ComicArchive, has_desired_tags, settings):
     md = GenericMetadata()
     md.set_default_page_list(ca.get_number_of_pages())
 
     # now, overlay the parsed filename info
     if opts.parse_filename:
-        md.overlay(ca.metadata_from_filename())
+        md.overlay(
+            ca.metadata_from_filename(
+                settings.complicated_parser, settings.remove_c2c, settings.remove_fcbd, settings.remove_publisher
+            )
+        )
 
     if has_desired_tags:
         md = ca.read_metadata(opts.data_style)
@@ -188,7 +192,7 @@ def process_file_cli(filename, opts, settings, match_results: OnlineMatchResults
     ca = ComicArchive(filename, settings.rar_exe_path, ComicTaggerSettings.get_graphic("nocover.png"))
 
     if not os.path.lexists(filename):
-        logger.error("Cannot find " + filename)
+        logger.error("Cannot find %s", filename)
         return
 
     if not ca.seems_to_be_a_comic_archive():
@@ -319,7 +323,7 @@ def process_file_cli(filename, opts, settings, match_results: OnlineMatchResults
         if batch_mode:
             print(f"Processing {ca.path}...")
 
-        md = create_local_metadata(opts, ca, has[opts.data_style])
+        md = create_local_metadata(opts, ca, has[opts.data_style], settings)
         if md.issue is None or md.issue == "":
             if opts.assume_issue_is_one_if_not_set:
                 md.issue = "1"
@@ -430,7 +434,7 @@ def process_file_cli(filename, opts, settings, match_results: OnlineMatchResults
         else:
             use_tags = False
 
-        md = create_local_metadata(opts, ca, use_tags)
+        md = create_local_metadata(opts, ca, use_tags, settings)
 
         if md.series is None:
             logger.error(msg_hdr + "Can't rename without series name")
@@ -445,24 +449,39 @@ def process_file_cli(filename, opts, settings, match_results: OnlineMatchResults
             elif ca.is_rar():
                 new_ext = ".cbr"
 
-        renamer = FileRenamer(md)
+        renamer = FileRenamer(md, platform="universal" if settings.rename_strict else "auto")
         renamer.set_template(settings.rename_template)
         renamer.set_issue_zero_padding(settings.rename_issue_number_padding)
         renamer.set_smart_cleanup(settings.rename_use_smart_string_cleanup)
+        renamer.move = settings.rename_move_dir
 
-        new_name = renamer.determine_name(ca.path, ext=new_ext)
-
-        if new_name == os.path.basename(ca.path):
-            logger.error(msg_hdr + "Filename is already good!")
+        try:
+            new_name = renamer.determine_name(ext=new_ext)
+        except Exception:
+            logger.exception(
+                msg_hdr + "Invalid format string!\n"
+                "Your rename template is invalid!\n\n"
+                "Please consult the template help in the settings "
+                "and the documentation on the format at "
+                "https://docs.python.org/3/library/string.html#format-string-syntax"
+            )
             return
 
-        folder = os.path.dirname(os.path.abspath(ca.path))
+        folder = os.path.dirname(os.path.abspath(filename))
+        if settings.rename_move_dir and len(settings.rename_dir.strip()) > 3:
+            folder = settings.rename_dir.strip()
+
         new_abs_path = utils.unique_file(os.path.join(folder, new_name))
 
+        if os.path.join(folder, new_name) == os.path.abspath(filename):
+            print(msg_hdr + "Filename is already good!", file=sys.stderr)
+            return
+
         suffix = ""
         if not opts.dryrun:
             # rename the file
-            os.rename(ca.path, new_abs_path)
+            os.makedirs(os.path.dirname(new_abs_path), 0o777, True)
+            os.rename(filename, new_abs_path)
         else:
             suffix = " (dry-run, no change)"
 

comictaggerlib/comicvinecacher.py

@@ -59,11 +59,11 @@ class ComicVineCacher:
     def create_cache_db(self):
 
         # create the version file
-        with open(self.version_file, "w") as f:
+        with open(self.version_file, "w", encoding="utf-8") as f:
             f.write(ctversion.version)
 
         # this will wipe out any existing version
-        open(self.db_file, "w").close()
+        open(self.db_file, "wb").close()
 
         con = lite.connect(self.db_file)
 
@@ -175,16 +175,15 @@ class ComicVineCacher:
             rows = cur.fetchall()
             # now process the results
             for record in rows:
-                result = {}
-                result["id"] = record[1]
-                result["name"] = record[2]
-                result["start_year"] = record[3]
-                result["publisher"] = {}
-                result["publisher"]["name"] = record[4]
-                result["count_of_issues"] = record[5]
-                result["image"] = {}
-                result["image"]["super_url"] = record[6]
-                result["description"] = record[7]
+                result = {
+                    "id": record[1],
+                    "name": record[2],
+                    "start_year": record[3],
+                    "count_of_issues": record[5],
+                    "description": record[7],
+                    "publisher": {"name": record[4]},
+                    "image": {"super_url": record[6]},
+                }
 
                 results.append(result)
 
@@ -301,16 +300,15 @@ class ComicVineCacher:
             if row is None:
                 return result
 
-            result = {}
-
             # since ID is primary key, there is only one row
-            result["id"] = row[0]
-            result["name"] = row[1]
-            result["publisher"] = {}
-            result["publisher"]["name"] = row[2]
-            result["count_of_issues"] = row[3]
-            result["start_year"] = row[4]
-            result["issues"] = []
+            result = {
+                "id": row[0],
+                "name": row[1],
+                "count_of_issues": row[3],
+                "start_year": row[4],
+                "issues": [],
+                "publisher": {"name": row[2]},
+            }
 
         return result
 
@@ -337,17 +335,15 @@ class ComicVineCacher:
 
             # now process the results
             for row in rows:
-                record = {}
-
-                record["id"] = row[0]
-                record["name"] = row[1]
-                record["issue_number"] = row[2]
-                record["site_detail_url"] = row[3]
-                record["cover_date"] = row[4]
-                record["image"] = {}
-                record["image"]["super_url"] = row[5]
-                record["image"]["thumb_url"] = row[6]
-                record["description"] = row[7]
+                record = {
+                    "id": row[0],
+                    "name": row[1],
+                    "issue_number": row[2],
+                    "site_detail_url": row[3],
+                    "cover_date": row[4],
+                    "image": {"super_url": row[5], "thumb_url": row[6]},
+                    "description": row[7],
+                }
 
                 results.append(record)
 

comictaggerlib/comicvinetalker.py

@@ -144,8 +144,7 @@ class ComicVineTalker:
 
             cv_response = requests.get(test_url, headers={"user-agent": "comictagger/" + ctversion.version}).json()
 
-            # Bogus request, but if the key is wrong, you get error 100: "Invalid
-            # API Key"
+            # Bogus request, but if the key is wrong, you get error 100: "Invalid API Key"
             return cv_response["status_code"] != 100
         except:
             return False
@@ -208,8 +207,7 @@ class ComicVineTalker:
         # Sanitize the series name for comicvine searching, comicvine search ignore symbols
         search_series_name = utils.sanitize_title(series_name)
 
-        # before we search online, look in our cache, since we might have
-        # done this same search recently
+        # before we search online, look in our cache, since we might have done this same search recently
         cvc = ComicVineCacher()
         if not refresh_cache:
             cached_search_results = cvc.get_search_results(series_name)
@@ -238,13 +236,11 @@ class ComicVineTalker:
 
         # 8 Dec 2018 - Comic Vine changed query results again. Terms are now
         # ORed together, and we get thousands of results.  Good news is the
-        # results are sorted by relevance, so we can be smart about halting
-        # the search.
+        # results are sorted by relevance, so we can be smart about halting the search.
         # 1. Don't fetch more than some sane amount of pages.
         max_results = 500
         # 2. Halt when not all of our search terms are present in a result
-        # 3. Halt when the results contain more (plus threshold) words than
-        #    our search
+        # 3. Halt when the results contain more (plus threshold) words than our search
         result_word_count_max = len(search_series_name.split()) + 3
 
         total_result_count = min(total_result_count, max_results)
@@ -516,17 +512,6 @@ class ComicVineTalker:
         return metadata
 
     def cleanup_html(self, string, remove_html_tables):
-        """
-        converter = html2text.HTML2Text()
-        #converter.emphasis_mark = '*'
-        #converter.ignore_links = True
-        converter.body_width = 0
-
-        print(html2text.html2text(string))
-        return string
-        #return converter.handle(string)
-        """
-
         if string is None:
             return ""
         # find any tables
@@ -538,9 +523,20 @@ class ComicVineTalker:
 
         # put in our own
         string = string.replace("<br>", "\n")
+        string = string.replace("</li>", "\n")
         string = string.replace("</p>", "\n\n")
+        string = string.replace("<h1>", "*")
+        string = string.replace("</h1>", "*\n")
+        string = string.replace("<h2>", "*")
+        string = string.replace("</h2>", "*\n")
+        string = string.replace("<h3>", "*")
+        string = string.replace("</h3>", "*\n")
         string = string.replace("<h4>", "*")
         string = string.replace("</h4>", "*\n")
+        string = string.replace("<h5>", "*")
+        string = string.replace("</h5>", "*\n")
+        string = string.replace("<h6>", "*")
+        string = string.replace("</h6>", "*\n")
 
         # remove the tables
         p = re.compile(r"<table[^<]*?>.*?</table>")
@@ -633,16 +629,12 @@ class ComicVineTalker:
 
         cv_response = self.get_cv_content(issue_url, params)
 
-        details: SelectDetails = {}
-        details["image_url"] = None
-        details["thumb_image_url"] = None
-        details["cover_date"] = None
-        details["site_detail_url"] = None
-
-        details["image_url"] = cv_response["results"]["image"]["super_url"]
-        details["thumb_image_url"] = cv_response["results"]["image"]["thumb_url"]
-        details["cover_date"] = cv_response["results"]["cover_date"]
-        details["site_detail_url"] = cv_response["results"]["site_detail_url"]
+        details: SelectDetails = {
+            "image_url": cv_response["results"]["image"]["super_url"],
+            "thumb_image_url": cv_response["results"]["image"]["thumb_url"],
+            "cover_date": cv_response["results"]["cover_date"],
+            "site_detail_url": cv_response["results"]["site_detail_url"],
+        }
 
         if details["image_url"] is not None:
             self.cache_issue_select_details(
@@ -656,8 +648,7 @@ class ComicVineTalker:
 
     def fetch_cached_issue_select_details(self, issue_id):
 
-        # before we search online, look in our cache, since we might already
-        # have this info
+        # before we search online, look in our cache, since we might already have this info
         cvc = ComicVineCacher()
         return cvc.get_issue_select_details(issue_id)
 
@@ -685,8 +676,7 @@ class ComicVineTalker:
         alt_cover_url_list = []
 
         # Using knowledge of the layout of the Comic Vine issue page here:
-        # look for the divs that are in the classes 'imgboxart' and
-        # 'issue-cover'
+        # look for the divs that are in the classes 'imgboxart' and 'issue-cover'
         div_list = soup.find_all("div")
         covers_found = 0
         for d in div_list:
@@ -706,8 +696,7 @@ class ComicVineTalker:
 
     def fetch_cached_alternate_cover_urls(self, issue_id):
 
-        # before we search online, look in our cache, since we might already
-        # have this info
+        # before we search online, look in our cache, since we might already have this info
         cvc = ComicVineCacher()
         url_list = cvc.get_alt_covers(issue_id)
         if url_list is not None:

comictaggerlib/filerenamer.py

@@ -14,10 +14,14 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-import datetime
+import calendar
 import logging
 import os
-import re
+import pathlib
+import string
+import sys
+
+from pathvalidate import sanitize_filename
 
 from comicapi.genericmetadata import GenericMetadata
 from comicapi.issuestring import IssueString
@@ -25,12 +29,94 @@ from comicapi.issuestring import IssueString
 logger = logging.getLogger(__name__)
 
 
+class MetadataFormatter(string.Formatter):
+    def __init__(self, smart_cleanup=False, platform="auto"):
+        super().__init__()
+        self.smart_cleanup = smart_cleanup
+        self.platform = platform
+
+    def format_field(self, value, format_spec):
+        if value is None or value == "":
+            return ""
+        return super().format_field(value, format_spec)
+
+    def _vformat(self, format_string, args, kwargs, used_args, recursion_depth, auto_arg_index=0):
+        if recursion_depth < 0:
+            raise ValueError("Max string recursion exceeded")
+        result = []
+        lstrip = False
+        for literal_text, field_name, format_spec, conversion in self.parse(format_string):
+
+            # output the literal text
+            if literal_text:
+                if lstrip:
+                    literal_text = literal_text.lstrip("-_)}]#")
+                if self.smart_cleanup:
+                    lspace = literal_text[0].isspace() if literal_text else False
+                    rspace = literal_text[-1].isspace() if literal_text else False
+                    literal_text = " ".join(literal_text.split())
+                    if literal_text == "":
+                        literal_text = " "
+                    else:
+                        if lspace:
+                            literal_text = " " + literal_text
+                        if rspace:
+                            literal_text += " "
+                result.append(literal_text)
+
+            lstrip = False
+            # if there's a field, output it
+            if field_name is not None and field_name != "":
+                field_name = field_name.lower()
+                # this is some markup, find the object and do the formatting
+
+                # handle arg indexing when empty field_names are given.
+                if field_name == "":
+                    if auto_arg_index is False:
+                        raise ValueError("cannot switch from manual field specification to automatic field numbering")
+                    field_name = str(auto_arg_index)
+                    auto_arg_index += 1
+                elif field_name.isdigit():
+                    if auto_arg_index:
+                        raise ValueError("cannot switch from manual field specification to automatic field numbering")
+                    # disable auto arg incrementing, if it gets used later on, then an exception will be raised
+                    auto_arg_index = False
+
+                # given the field_name, find the object it references
+                #  and the argument it came from
+                obj, arg_used = self.get_field(field_name, args, kwargs)
+                used_args.add(arg_used)
+
+                # do any conversion on the resulting object
+                obj = self.convert_field(obj, conversion)
+
+                # expand the format spec, if needed
+                format_spec, auto_arg_index = self._vformat(
+                    format_spec, args, kwargs, used_args, recursion_depth - 1, auto_arg_index=auto_arg_index
+                )
+
+                # format the object and append to the result
+                fmt_obj = self.format_field(obj, format_spec)
+                if fmt_obj == "" and len(result) > 0 and self.smart_cleanup:
+                    lstrip = True
+                    if result:
+                        result[-1] = result[-1].rstrip("-_({[#")
+                if self.smart_cleanup:
+                    fmt_obj = " ".join(fmt_obj.split())
+                    fmt_obj = sanitize_filename(fmt_obj, platform=self.platform)
+                result.append(fmt_obj)
+
+        return "".join(result), auto_arg_index
+
+
 class FileRenamer:
-    def __init__(self, metadata):
-        self.template = "%series% v%volume% #%issue% (of %issuecount%) (%year%)"
+    def __init__(self, metadata, platform="auto"):
+        self.template = "{publisher}/{series}/{series} v{volume} #{issue} (of {issue_count}) ({year})"
         self.smart_cleanup = True
         self.issue_zero_padding = 3
         self.metadata = metadata
+        self.move = False
+        self.platform = platform
 
     def set_metadata(self, metadata: GenericMetadata):
         self.metadata = metadata
@@ -44,101 +130,50 @@ class FileRenamer:
     def set_template(self, template: str):
         self.template = template
 
-    def replace_token(self, text, value, token):
-        # helper func
-        def is_token(word):
-            return word[0] == "%" and word[-1:] == "%"
-
-        if value is not None:
-            return text.replace(token, str(value))
-
-        if self.smart_cleanup:
-            # smart cleanup means we want to remove anything appended to token if it's empty (e.g "#%issue%"  or "v%volume%")
-            # (TODO: This could fail if there is more than one token appended together, I guess)
-            text_list = text.split()
-
-            # special case for issuecount, remove preceding non-token word,
-            # as in "...(of %issuecount%)..."
-            if token == "%issuecount%":
-                for idx, word in enumerate(text_list):
-                    if token in word and not is_token(text_list[idx - 1]):
-                        text_list[idx - 1] = ""
-
-            text_list = [x for x in text_list if token not in x]
-            return " ".join(text_list)
-
-        return text.replace(token, "")
-
-    def determine_name(self, filename, ext=None):
+    def determine_name(self, ext):
+        class Default(dict):
+            def __missing__(self, key):
+                return "{" + key + "}"
 
         md = self.metadata
-        new_name = self.template
 
-        new_name = self.replace_token(new_name, md.series, "%series%")
-        new_name = self.replace_token(new_name, md.volume, "%volume%")
+        # padding for issue
+        md.issue = IssueString(md.issue).as_string(pad=self.issue_zero_padding)
 
-        if md.issue is not None:
-            issue_str = IssueString(md.issue).as_string(pad=self.issue_zero_padding)
+        template = self.template
+
+        new_name = ""
+
+        fmt = MetadataFormatter(self.smart_cleanup, platform=self.platform)
+        md_dict = vars(md)
+        for role in ["writer", "penciller", "inker", "colorist", "letterer", "cover artist", "editor"]:
+            md_dict[role] = md.get_primary_credit(role)
+
+        if (isinstance(md.month, int) or isinstance(md.month, str) and md.month.isdigit()) and 0 < int(md.month) < 13:
+            md_dict["month_name"] = calendar.month_name[int(md.month)]
+            md_dict["month_abbr"] = calendar.month_abbr[int(md.month)]
         else:
-            issue_str = None
-        new_name = self.replace_token(new_name, issue_str, "%issue%")
+            md_dict["month_name"] = ""
+            md_dict["month_abbr"] = ""
 
-        new_name = self.replace_token(new_name, md.issue_count, "%issuecount%")
-        new_name = self.replace_token(new_name, md.year, "%year%")
-        new_name = self.replace_token(new_name, md.publisher, "%publisher%")
-        new_name = self.replace_token(new_name, md.title, "%title%")
-        new_name = self.replace_token(new_name, md.month, "%month%")
-        month_name = None
-        if md.month is not None:
-            if (isinstance(md.month, str) and md.month.isdigit()) or isinstance(md.month, int):
-                if int(md.month) in range(1, 13):
-                    dt = datetime.datetime(1970, int(md.month), 1, 0, 0)
-                    month_name = dt.strftime("%B")
-        new_name = self.replace_token(new_name, month_name, "%month_name%")
+        for Component in pathlib.PureWindowsPath(template).parts:
+            if (
+                self.platform.lower() in ["universal", "windows"] or sys.platform.lower() in ["windows"]
+            ) and self.smart_cleanup:
+                # colons get special treatment
+                Component = Component.replace(": ", " - ")
+                Component = Component.replace(":", "-")
 
-        new_name = self.replace_token(new_name, md.genre, "%genre%")
-        new_name = self.replace_token(new_name, md.language, "%language_code%")
-        new_name = self.replace_token(new_name, md.critical_rating, "%criticalrating%")
-        new_name = self.replace_token(new_name, md.alternate_series, "%alternateseries%")
-        new_name = self.replace_token(new_name, md.alternate_number, "%alternatenumber%")
-        new_name = self.replace_token(new_name, md.alternate_count, "%alternatecount%")
-        new_name = self.replace_token(new_name, md.imprint, "%imprint%")
-        new_name = self.replace_token(new_name, md.format, "%format%")
-        new_name = self.replace_token(new_name, md.maturity_rating, "%maturityrating%")
-        new_name = self.replace_token(new_name, md.story_arc, "%storyarc%")
-        new_name = self.replace_token(new_name, md.series_group, "%seriesgroup%")
-        new_name = self.replace_token(new_name, md.scan_info, "%scaninfo%")
-
-        if self.smart_cleanup:
-            # remove empty braces,brackets, parentheses
-            new_name = re.sub(r"\(\s*[-:]*\s*\)", "", new_name)
-            new_name = re.sub(r"\[\s*[-:]*\s*]", "", new_name)
-            new_name = re.sub(r"{\s*[-:]*\s*}", "", new_name)
-
-            # remove duplicate spaces
-            new_name = " ".join(new_name.split())
-
-            # remove remove duplicate -, _,
-            new_name = re.sub(r"[-_]{2,}\s+", "-- ", new_name)
-            new_name = re.sub(r"(\s--)+", " --", new_name)
-            new_name = re.sub(r"(\s-)+", " -", new_name)
-
-            # remove dash or double dash at end of line
-            new_name = re.sub(r"[-]{1,2}\s*$", "", new_name)
-
-            # remove duplicate spaces (again!)
-            new_name = " ".join(new_name.split())
-
-        if ext is None:
-            ext = os.path.splitext(filename)[1]
+            new_basename = sanitize_filename(
+                fmt.vformat(Component, args=None, kwargs=Default(md_dict)), platform=self.platform
+            ).strip()
+            new_name = os.path.join(new_name, new_basename)
 
         new_name += ext
+        new_basename += ext
 
-        # some tweaks to keep various filesystems happy
-        new_name = new_name.replace("/", "-")
-        new_name = new_name.replace(" :", " -")
-        new_name = new_name.replace(": ", " - ")
-        new_name = new_name.replace(":", "-")
-        new_name = new_name.replace("?", "")
-
-        return new_name
+        # remove padding
+        md.issue = IssueString(md.issue).as_string()
+        if self.move:
+            return new_name.strip()
+        return new_basename.strip()

comictaggerlib/imagefetcher.py

@@ -15,6 +15,7 @@
 # limitations under the License.
 
 import datetime
+import logging
 import os
 import shutil
 import sqlite3 as lite
@@ -22,6 +23,9 @@ import tempfile
 
 import requests
 
+from comictaggerlib import ctversion
+from comictaggerlib.settings import ComicTaggerSettings
+
 try:
     from PyQt5 import QtCore, QtNetwork
 
@@ -29,12 +33,6 @@ try:
 except ImportError:
     qt_available = False
 
-
-import logging
-
-from comictaggerlib import ctversion
-from comictaggerlib.settings import ComicTaggerSettings
-
 logger = logging.getLogger(__name__)
 
 
@@ -120,7 +118,7 @@ class ImageFetcher:
     def create_image_db(self):
 
         # this will wipe out any existing version
-        open(self.db_file, "w").close()
+        open(self.db_file, "wb").close()
 
         # wipe any existing image cache folder too
         if os.path.isdir(self.cache_folder):
@@ -145,9 +143,8 @@ class ImageFetcher:
             timestamp = datetime.datetime.now()
 
             tmp_fd, filename = tempfile.mkstemp(dir=self.cache_folder, prefix="img")
-            f = os.fdopen(tmp_fd, "w+b")
-            f.write(image_data)
-            f.close()
+            with os.fdopen(tmp_fd, "w+b") as f:
+                f.write(image_data)
 
             cur.execute("INSERT or REPLACE INTO Images VALUES(?, ?, ?)", (url, filename, timestamp))
 

comictaggerlib/imagehasher.py

@@ -72,8 +72,7 @@ class ImageHasher:
 
     def average_hash2(self):
         """
-        # Got this one from somewhere on the net.  Not a clue how the 'convolve2d'
-        # works!
+        # Got this one from somewhere on the net.  Not a clue how the 'convolve2d' works!
 
         from numpy import array
         from scipy.signal import convolve2d

comictaggerlib/issueidentifier.py

@@ -63,6 +63,7 @@ class IssueIdentifier:
     result_multiple_good_matches = 5
 
     def __init__(self, comic_archive: ComicArchive, settings):
+        self.settings = settings
         self.comic_archive: ComicArchive = comic_archive
         self.image_hasher = 1
 
@@ -112,8 +113,8 @@ class IssueIdentifier:
     def set_name_length_delta_threshold(self, delta):
         self.length_delta_thresh = delta
 
-    def set_publisher_filter(self, filter):
-        self.publisher_filter = filter
+    def set_publisher_filter(self, flt):
+        self.publisher_filter = flt
 
     def set_hasher_algorithm(self, algo):
         self.image_hasher = algo
@@ -164,12 +165,13 @@ class IssueIdentifier:
     def get_search_keys(self):
 
         ca = self.comic_archive
-        search_keys: SearchKeys = {}
-        search_keys["series"] = None
-        search_keys["issue_number"] = None
-        search_keys["month"] = None
-        search_keys["year"] = None
-        search_keys["issue_count"] = None
+        search_keys: SearchKeys = {
+            "series": None,
+            "issue_number": None,
+            "month": None,
+            "year": None,
+            "issue_count": None,
+        }
 
         if ca is None:
             return None
@@ -191,7 +193,12 @@ class IssueIdentifier:
             internal_metadata = ca.read_cbi()
 
         # try to get some metadata from filename
-        md_from_filename = ca.metadata_from_filename()
+        md_from_filename = ca.metadata_from_filename(
+            self.settings.complicated_parser,
+            self.settings.remove_c2c,
+            self.settings.remove_fcbd,
+            self.settings.remove_publisher,
+        )
 
         # preference order:
         # 1. Additional metadata
@@ -274,10 +281,8 @@ class IssueIdentifier:
             self.cover_url_callback(url_image_data)
 
         remote_cover_list = []
-        item = {}
-        item["url"] = primary_img_url
+        item = {"url": primary_img_url, "hash": self.calculate_hash(url_image_data)}
 
-        item["hash"] = self.calculate_hash(url_image_data)
         remote_cover_list.append(item)
 
         if self.cancel:
@@ -299,9 +304,7 @@ class IssueIdentifier:
                 if self.cover_url_callback is not None:
                     self.cover_url_callback(alt_url_image_data)
 
-                item = {}
-                item["url"] = alt_url
-                item["hash"] = self.calculate_hash(alt_url_image_data)
+                item = {"url": alt_url, "hash": self.calculate_hash(alt_url_image_data)}
                 remote_cover_list.append(item)
 
                 if self.cancel:
@@ -317,10 +320,7 @@ class IssueIdentifier:
         for local_cover_hash in local_cover_hash_list:
             for remote_cover_item in remote_cover_list:
                 score = ImageHasher.hamming_distance(local_cover_hash, remote_cover_item["hash"])
-                score_item = {}
-                score_item["score"] = score
-                score_item["url"] = remote_cover_item["url"]
-                score_item["hash"] = remote_cover_item["hash"]
+                score_item = {"score": score, "url": remote_cover_item["url"], "hash": remote_cover_item["hash"]}
                 score_list.append(score_item)
                 if use_log:
                     self.log_msg(score, False)
@@ -520,24 +520,25 @@ class IssueIdentifier:
                 self.match_list = []
                 return self.match_list
 
-            match: IssueResult = {}
-            match["series"] = f"{series['name']} ({series['start_year']})"
-            match["distance"] = score_item["score"]
-            match["issue_number"] = keys["issue_number"]
-            match["cv_issue_count"] = series["count_of_issues"]
-            match["url_image_hash"] = score_item["hash"]
-            match["issue_title"] = issue["name"]
-            match["issue_id"] = issue["id"]
-            match["volume_id"] = series["id"]
-            match["month"] = month
-            match["year"] = year
-            match["publisher"] = None
+            match: IssueResult = {
+                "series": f"{series['name']} ({series['start_year']})",
+                "distance": score_item["score"],
+                "issue_number": keys["issue_number"],
+                "cv_issue_count": series["count_of_issues"],
+                "url_image_hash": score_item["hash"],
+                "issue_title": issue["name"],
+                "issue_id": issue["id"],
+                "volume_id": series["id"],
+                "month": month,
+                "year": year,
+                "publisher": None,
+                "image_url": image_url,
+                "thumb_url": thumb_url,
+                "page_url": page_url,
+                "description": issue["description"],
+            }
             if series["publisher"] is not None:
                 match["publisher"] = series["publisher"]["name"]
-            match["image_url"] = image_url
-            match["thumb_url"] = thumb_url
-            match["page_url"] = page_url
-            match["description"] = issue["description"]
 
             self.match_list.append(match)
 
@@ -562,7 +563,7 @@ class IssueIdentifier:
 
         def print_match(item):
             self.log_msg(
-                "-----> {0} #{1} {2} ({3}/{4}) -- score: {5}".format(
+                "-----> {} #{} {} ({}/{}) -- score: {}".format(
                     item["series"],
                     item["issue_number"],
                     item["issue_title"],

comictaggerlib/logwindow.py

@@ -20,6 +20,7 @@ import logging
 from PyQt5 import QtCore, QtWidgets, uic
 
 from comictaggerlib.settings import ComicTaggerSettings
+from comictaggerlib.ui import qtutils
 
 logger = logging.getLogger(__name__)
 
@@ -41,6 +42,9 @@ class LogWindow(QtWidgets.QDialog):
     def set_text(self, text):
         try:
             text = text.decode()
-        except:
+            self.textEdit.setPlainText(text)
+        except AttributeError:
             pass
-        self.textEdit.setPlainText(text)
+        except Exception as e:
+            logger.exception("Displaying raw tags failed")
+            qtutils.qt_error("Displaying raw tags failed:", e)

comictaggerlib/main.py

@@ -37,11 +37,51 @@ logger.setLevel(logging.DEBUG)
 
 try:
     qt_available = True
-    from PyQt5 import QtGui, QtWidgets
+    from PyQt5 import QtCore, QtGui, QtWidgets
 
+    def show_exception_box(log_msg):
+        """Checks if a QApplication instance is available and shows a messagebox with the exception message.
+        If unavailable (non-console application), log an additional notice.
+        """
+        if QtWidgets.QApplication.instance() is not None:
+            errorbox = QtWidgets.QMessageBox()
+            errorbox.setText(f"Oops. An unexpected error occured:\n{log_msg}")
+            errorbox.exec()
+            QtWidgets.QApplication.exit(1)
+        else:
+            logger.debug("No QApplication instance available.")
+
+    class UncaughtHook(QtCore.QObject):
+        _exception_caught = QtCore.pyqtSignal(object)
+
+        def __init__(self, *args, **kwargs):
+            super().__init__(*args, **kwargs)
+
+            # this registers the exception_hook() function as hook with the Python interpreter
+            sys.excepthook = self.exception_hook
+
+            # connect signal to execute the message box function always on main thread
+            self._exception_caught.connect(show_exception_box)
+
+        def exception_hook(self, exc_type, exc_value, exc_traceback):
+            """Function handling uncaught exceptions.
+            It is triggered each time an uncaught exception occurs.
+            """
+            if issubclass(exc_type, KeyboardInterrupt):
+                # ignore keyboard interrupt to support console applications
+                sys.__excepthook__(exc_type, exc_value, exc_traceback)
+            else:
+                exc_info = (exc_type, exc_value, exc_traceback)
+                log_msg = "\n".join(["".join(traceback.format_tb(exc_traceback)), f"{exc_type.__name__}: {exc_value}"])
+                logger.critical("Uncaught exception: %s: %s", exc_type.__name__, exc_value, exc_info=exc_info)
+
+                # trigger message box show
+                self._exception_caught.emit(log_msg)
+
+    qt_exception_hook = UncaughtHook()
     from comictaggerlib.taggerwindow import TaggerWindow
 except ImportError as e:
-    logging.debug(e)
+    logger.error(str(e))
     qt_available = False
 
 
@@ -51,6 +91,10 @@ def rotate(handler: logging.handlers.RotatingFileHandler, filename: pathlib.Path
 
 
 def ctmain():
+    opts = Options()
+    opts.parse_cmd_line_args()
+    SETTINGS = ComicTaggerSettings(opts.config_path)
+
     os.makedirs(ComicTaggerSettings.get_settings_folder() / "logs", exist_ok=True)
     stream_handler = logging.StreamHandler()
     stream_handler.setLevel(logging.WARNING)
@@ -67,11 +111,7 @@ def ctmain():
         format="%(asctime)s | %(name)s | %(levelname)s | %(message)s",
         datefmt="%Y-%m-%dT%H:%M:%S",
     )
-    opts = Options()
-    opts.parse_cmd_line_args()
-
     # Need to load setting before anything else
-    SETTINGS = ComicTaggerSettings()
 
     # manage the CV API key
     if opts.cv_api_key:
@@ -106,7 +146,7 @@ def ctmain():
         try:
             cli.cli_mode(opts, SETTINGS)
         except:
-            logger.exception()
+            logger.exception("CLI mode failed")
     else:
         os.environ["QtWidgets.QT_AUTO_SCREEN_SCALE_FACTOR"] = "1"
         args = []
@@ -149,7 +189,7 @@ def ctmain():
 
             sys.exit(app.exec())
         except Exception:
-            logger.exception()
+            logger.exception("GUI mode failed")
             QtWidgets.QMessageBox.critical(
                 QtWidgets.QMainWindow(), "Error", "Unhandled exception in app:\n" + traceback.format_exc()
             )

comictaggerlib/options.py

@@ -241,7 +241,7 @@ For more help visit the wiki at: https://github.com/comictagger/comictagger/wiki
             input_args = sys.argv[1:]
 
         # first check if we're launching a script:
-        for n in range(len(input_args)):
+        for n, _ in enumerate(input_args):
             if input_args[n] in ["-S", "--script"] and n + 1 < len(input_args):
                 # insert a "--" which will cause getopt to ignore the remaining args
                 # so they will be passed to the script

comictaggerlib/pagelisteditor.py

@@ -35,12 +35,9 @@ def item_move_events(widget):
         def eventFilter(self, obj, event):
 
             if obj == widget:
-                # print(event.type())
                 if event.type() == QtCore.QEvent.Type.ChildRemoved:
-                    # print("ChildRemoved")
                     self.mysignal.emit("finish")
                 if event.type() == QtCore.QEvent.Type.ChildAdded:
-                    # print("ChildAdded")
                     self.mysignal.emit("start")
                     return True
 
@@ -83,23 +80,24 @@ class PageListEditor(QtWidgets.QWidget):
 
         self.reset_page()
 
-        # Add the entries to the manga combobox
-        self.comboBox.addItem("", "")
-        self.comboBox.addItem(self.pageTypeNames[PageType.FrontCover], PageType.FrontCover)
-        self.comboBox.addItem(self.pageTypeNames[PageType.InnerCover], PageType.InnerCover)
-        self.comboBox.addItem(self.pageTypeNames[PageType.Advertisement], PageType.Advertisement)
-        self.comboBox.addItem(self.pageTypeNames[PageType.Roundup], PageType.Roundup)
-        self.comboBox.addItem(self.pageTypeNames[PageType.Story], PageType.Story)
-        self.comboBox.addItem(self.pageTypeNames[PageType.Editorial], PageType.Editorial)
-        self.comboBox.addItem(self.pageTypeNames[PageType.Letters], PageType.Letters)
-        self.comboBox.addItem(self.pageTypeNames[PageType.Preview], PageType.Preview)
-        self.comboBox.addItem(self.pageTypeNames[PageType.BackCover], PageType.BackCover)
-        self.comboBox.addItem(self.pageTypeNames[PageType.Other], PageType.Other)
-        self.comboBox.addItem(self.pageTypeNames[PageType.Deleted], PageType.Deleted)
+        # Add the entries to the page type combobox
+        self.add_page_type_item("", "", "Alt+0", False)
+        self.add_page_type_item(self.pageTypeNames[PageType.FrontCover], PageType.FrontCover, "Alt+F")
+        self.add_page_type_item(self.pageTypeNames[PageType.InnerCover], PageType.InnerCover, "Alt+I")
+        self.add_page_type_item(self.pageTypeNames[PageType.Advertisement], PageType.Advertisement, "Alt+A")
+        self.add_page_type_item(self.pageTypeNames[PageType.Roundup], PageType.Roundup, "Alt+R")
+        self.add_page_type_item(self.pageTypeNames[PageType.Story], PageType.Story, "Alt+S")
+        self.add_page_type_item(self.pageTypeNames[PageType.Editorial], PageType.Editorial, "Alt+E")
+        self.add_page_type_item(self.pageTypeNames[PageType.Letters], PageType.Letters, "Alt+L")
+        self.add_page_type_item(self.pageTypeNames[PageType.Preview], PageType.Preview, "Alt+P")
+        self.add_page_type_item(self.pageTypeNames[PageType.BackCover], PageType.BackCover, "Alt+B")
+        self.add_page_type_item(self.pageTypeNames[PageType.Other], PageType.Other, "Alt+O")
+        self.add_page_type_item(self.pageTypeNames[PageType.Deleted], PageType.Deleted, "Alt+X")
 
         self.listWidget.itemSelectionChanged.connect(self.change_page)
         item_move_events(self.listWidget).connect(self.item_move_event)
-        self.comboBox.activated.connect(self.change_page_type)
+        self.cbPageType.activated.connect(self.change_page_type)
+        self.chkDoublePage.toggled.connect(self.toggle_double_page)
         self.leBookmark.editingFinished.connect(self.save_bookmark)
         self.btnUp.clicked.connect(self.move_current_up)
         self.btnDown.clicked.connect(self.move_current_down)
@@ -111,11 +109,27 @@ class PageListEditor(QtWidgets.QWidget):
 
     def reset_page(self):
         self.pageWidget.clear()
-        self.comboBox.setDisabled(True)
+        self.cbPageType.setDisabled(True)
+        self.chkDoublePage.setDisabled(True)
         self.leBookmark.setDisabled(True)
         self.comic_archive = None
         self.pages_list = []
 
+    def add_page_type_item(self, text, user_data, shortcut, show_shortcut=True):
+        if show_shortcut:
+            text = text + " (" + shortcut + ")"
+        self.cbPageType.addItem(text, user_data)
+        actionItem = QtWidgets.QAction(
+            shortcut, self, triggered=lambda: self.select_page_type_item(self.cbPageType.findData(user_data))
+        )
+        actionItem.setShortcut(shortcut)
+        self.addAction(actionItem)
+
+    def select_page_type_item(self, idx):
+        if self.cbPageType.isEnabled():
+            self.cbPageType.setCurrentIndex(idx)
+            self.change_page_type(idx)
+
     def get_new_indexes(self, movement):
         selection = self.listWidget.selectionModel().selectedRows()
         selection.sort(reverse=movement > 0)
@@ -195,7 +209,7 @@ class PageListEditor(QtWidgets.QWidget):
                 self.modified.emit()
 
     def change_page_type(self, i):
-        new_type = self.comboBox.itemData(i)
+        new_type = self.cbPageType.itemData(i)
         if self.get_current_page_type() != new_type:
             self.set_current_page_type(new_type)
             self.emit_front_cover_change()
@@ -205,8 +219,10 @@ class PageListEditor(QtWidgets.QWidget):
         row = self.listWidget.currentRow()
         pagetype = self.get_current_page_type()
 
-        i = self.comboBox.findData(pagetype)
-        self.comboBox.setCurrentIndex(i)
+        i = self.cbPageType.findData(pagetype)
+        self.cbPageType.setCurrentIndex(i)
+
+        self.chkDoublePage.setChecked("DoublePage" in self.listWidget.item(row).data(QtCore.Qt.UserRole)[0])
 
         if "Bookmark" in self.listWidget.item(row).data(QtCore.Qt.UserRole)[0]:
             self.leBookmark.setText(self.listWidget.item(row).data(QtCore.Qt.UserRole)[0]["Bookmark"])
@@ -222,7 +238,7 @@ class PageListEditor(QtWidgets.QWidget):
         front_cover = 0
         for i in range(self.listWidget.count()):
             item = self.listWidget.item(i)
-            page_dict = item.data(QtCore.Qt.ItemDataRole.UserRole)[0]  # .toPyObject()[0]
+            page_dict = item.data(QtCore.Qt.ItemDataRole.UserRole)[0]
             if "Type" in page_dict and page_dict["Type"] == PageType.FrontCover:
                 front_cover = int(page_dict["Image"])
                 break
@@ -230,7 +246,7 @@ class PageListEditor(QtWidgets.QWidget):
 
     def get_current_page_type(self):
         row = self.listWidget.currentRow()
-        page_dict = self.listWidget.item(row).data(QtCore.Qt.ItemDataRole.UserRole)[0]  # .toPyObject()[0]
+        page_dict = self.listWidget.item(row).data(QtCore.Qt.ItemDataRole.UserRole)[0]
         if "Type" in page_dict:
             return page_dict["Type"]
 
@@ -238,19 +254,36 @@ class PageListEditor(QtWidgets.QWidget):
 
     def set_current_page_type(self, t):
         row = self.listWidget.currentRow()
-        page_dict = self.listWidget.item(row).data(QtCore.Qt.ItemDataRole.UserRole)[0]  # .toPyObject()[0]
+        page_dict = self.listWidget.item(row).data(QtCore.Qt.ItemDataRole.UserRole)[0]
 
         if t == "":
             if "Type" in page_dict:
                 del page_dict["Type"]
         else:
-            page_dict["Type"] = str(t)
+            page_dict["Type"] = t
 
         item = self.listWidget.item(row)
         # wrap the dict in a tuple to keep from being converted to QtWidgets.QStrings
         item.setData(QtCore.Qt.ItemDataRole.UserRole, (page_dict,))
         item.setText(self.list_entry_text(page_dict))
 
+    def toggle_double_page(self):
+        row = self.listWidget.currentRow()
+        page_dict = self.listWidget.item(row).data(QtCore.Qt.UserRole)[0]
+
+        if self.sender().isChecked():
+            page_dict["DoublePage"] = str("true")
+        elif "DoublePage" in page_dict:
+            del page_dict["DoublePage"]
+        self.modified.emit()
+
+        item = self.listWidget.item(row)
+        # wrap the dict in a tuple to keep from being converted to QStrings
+        item.setData(QtCore.Qt.UserRole, (page_dict,))
+        item.setText(self.list_entry_text(page_dict))
+
+        self.listWidget.setFocus()
+
     def save_bookmark(self):
         row = self.listWidget.currentRow()
         page_dict = self.listWidget.item(row).data(QtCore.Qt.UserRole)[0]
@@ -279,7 +312,8 @@ class PageListEditor(QtWidgets.QWidget):
         self.comic_archive = comic_archive
         self.pages_list = pages_list
         if pages_list is not None and len(pages_list) > 0:
-            self.comboBox.setDisabled(False)
+            self.cbPageType.setDisabled(False)
+            self.chkDoublePage.setDisabled(False)
             self.leBookmark.setDisabled(False)
 
         self.listWidget.itemSelectionChanged.disconnect(self.change_page)
@@ -298,10 +332,12 @@ class PageListEditor(QtWidgets.QWidget):
     def list_entry_text(self, page_dict):
         text = str(int(page_dict["Image"]) + 1)
         if "Type" in page_dict:
-            if page_dict["Type"] in self.pageTypeNames.keys():
+            if page_dict["Type"] in self.pageTypeNames:
                 text += " (" + self.pageTypeNames[page_dict["Type"]] + ")"
             else:
                 text += " (Error: " + page_dict["Type"] + ")"
+        if "DoublePage" in page_dict:
+            text += " " + "\U00002461"
         if "Bookmark" in page_dict:
             text += " " + "\U0001F516"
         return text
@@ -310,7 +346,7 @@ class PageListEditor(QtWidgets.QWidget):
         page_list = []
         for i in range(self.listWidget.count()):
             item = self.listWidget.item(i)
-            page_list.append(item.data(QtCore.Qt.ItemDataRole.UserRole)[0])  # .toPyObject()[0]
+            page_list.append(item.data(QtCore.Qt.ItemDataRole.UserRole)[0])
         return page_list
 
     def emit_front_cover_change(self):
@@ -322,15 +358,16 @@ class PageListEditor(QtWidgets.QWidget):
         # depending on the current data style, certain fields are disabled
 
         inactive_color = QtGui.QColor(255, 170, 150)
-        active_palette = self.comboBox.palette()
+        active_palette = self.cbPageType.palette()
 
-        inactive_palette3 = self.comboBox.palette()
+        inactive_palette3 = self.cbPageType.palette()
         inactive_palette3.setColor(QtGui.QPalette.ColorRole.Base, inactive_color)
 
         if data_style == MetaDataStyle.CIX:
             self.btnUp.setEnabled(True)
             self.btnDown.setEnabled(True)
-            self.comboBox.setEnabled(True)
+            self.cbPageType.setEnabled(True)
+            self.chkDoublePage.setEnabled(True)
             self.leBookmark.setEnabled(True)
             self.listWidget.setEnabled(True)
 
@@ -340,7 +377,8 @@ class PageListEditor(QtWidgets.QWidget):
         elif data_style == MetaDataStyle.CBI:
             self.btnUp.setEnabled(False)
             self.btnDown.setEnabled(False)
-            self.comboBox.setEnabled(False)
+            self.cbPageType.setEnabled(False)
+            self.chkDoublePage.setEnabled(False)
             self.leBookmark.setEnabled(False)
             self.listWidget.setEnabled(False)
 
@@ -352,5 +390,6 @@ class PageListEditor(QtWidgets.QWidget):
 
         # make sure combo is disabled when no list
         if self.comic_archive is None:
-            self.comboBox.setEnabled(False)
+            self.cbPageType.setEnabled(False)
+            self.chkDoublePage.setEnabled(False)
             self.leBookmark.setEnabled(False)

comictaggerlib/renamewindow.py

@@ -52,7 +52,8 @@ class RenameWindow(QtWidgets.QDialog):
         self.rename_list = []
 
         self.btnSettings.clicked.connect(self.modify_settings)
-        self.renamer = FileRenamer(None)
+        self.renamer = FileRenamer(None, platform="universal" if self.settings.rename_strict else "auto")
+
         self.config_renamer()
         self.do_preview()
 
@@ -80,9 +81,29 @@ class RenameWindow(QtWidgets.QDialog):
 
             md = ca.read_metadata(self.data_style)
             if md.is_empty:
-                md = ca.metadata_from_filename(self.settings.parse_scan_info)
+                md = ca.metadata_from_filename(
+                    self.settings.complicated_parser,
+                    self.settings.remove_c2c,
+                    self.settings.remove_fcbd,
+                    self.settings.remove_publisher,
+                )
             self.renamer.set_metadata(md)
-            new_name = self.renamer.determine_name(ca.path, ext=new_ext)
+            self.renamer.move = self.settings.rename_move_dir
+
+            try:
+                new_name = self.renamer.determine_name(new_ext)
+            except Exception as e:
+                QtWidgets.QMessageBox.critical(
+                    self,
+                    "Invalid format string!",
+                    "Your rename template is invalid!"
+                    f"<br/><br/>{e}<br/><br/>"
+                    "Please consult the template help in the "
+                    "settings and the documentation on the format at "
+                    "<a href='https://docs.python.org/3/library/string.html#format-string-syntax'>"
+                    "https://docs.python.org/3/library/string.html#format-string-syntax</a>",
+                )
+                return
 
             row = self.twList.rowCount()
             self.twList.insertRow(row)
@@ -150,17 +171,20 @@ class RenameWindow(QtWidgets.QDialog):
             center_window_on_parent(prog_dialog)
             QtCore.QCoreApplication.processEvents()
 
-            if item["new_name"] == os.path.basename(item["archive"].path):
-                print(item["new_name"], "Filename is already good!")
+            folder = os.path.dirname(os.path.abspath(item["archive"].path))
+            if self.settings.rename_move_dir and len(self.settings.rename_dir.strip()) > 3:
+                folder = self.settings.rename_dir.strip()
+
+            new_abs_path = utils.unique_file(os.path.join(folder, item["new_name"]))
+
+            if os.path.join(folder, item["new_name"]) == item["archive"].path:
                 logger.info(item["new_name"], "Filename is already good!")
                 continue
 
             if not item["archive"].is_writable(check_rar_status=False):
                 continue
 
-            folder = os.path.dirname(os.path.abspath(item["archive"].path))
-            new_abs_path = utils.unique_file(os.path.join(folder, item["new_name"]))
-
+            os.makedirs(os.path.dirname(new_abs_path), 0o777, True)
             os.rename(item["archive"].path, new_abs_path)
 
             item["archive"].rename(new_abs_path)

comictaggerlib/settings.py

@@ -28,13 +28,16 @@ logger = logging.getLogger(__name__)
 
 
 class ComicTaggerSettings:
+    folder = ""
+
     @staticmethod
     def get_settings_folder():
-        if platform.system() == "Windows":
-            folder = os.path.join(os.environ["APPDATA"], "ComicTagger")
-        else:
-            folder = os.path.join(os.path.expanduser("~"), ".ComicTagger")
-        return pathlib.Path(folder)
+        if not ComicTaggerSettings.folder:
+            if platform.system() == "Windows":
+                ComicTaggerSettings.folder = os.path.join(os.environ["APPDATA"], "ComicTagger")
+            else:
+                ComicTaggerSettings.folder = os.path.join(os.path.expanduser("~"), ".ComicTagger")
+        return pathlib.Path(ComicTaggerSettings.folder)
 
     @staticmethod
     def base_dir():
@@ -85,7 +88,10 @@ class ComicTaggerSettings:
         self.ask_about_usage_stats = True
 
         # filename parsing settings
-        self.parse_scan_info = True
+        self.complicated_parser = False
+        self.remove_c2c = False
+        self.remove_fcbd = False
+        self.remove_publisher = False
 
         # Comic Vine settings
         self.use_series_start_as_volume = False
@@ -114,6 +120,9 @@ class ComicTaggerSettings:
         self.rename_issue_number_padding = 3
         self.rename_use_smart_string_cleanup = True
         self.rename_extension_based_on_archive = True
+        self.rename_dir = ""
+        self.rename_move_dir = False
+        self.rename_strict = False
 
         # Auto-tag stickies
         self.save_on_low_confidence = False
@@ -123,10 +132,7 @@ class ComicTaggerSettings:
         self.remove_archive_after_successful_match = False
         self.wait_and_retry_on_rate_limit = False
 
-    def __init__(self):
-
-        self.settings_file = ""
-        self.folder = ""
+    def __init__(self, folder):
         # General Settings
         self.rar_exe_path = ""
         self.allow_cbi_in_rar = True
@@ -158,7 +164,10 @@ class ComicTaggerSettings:
         self.ask_about_usage_stats = True
 
         # filename parsing settings
-        self.parse_scan_info = True
+        self.complicated_parser = False
+        self.remove_c2c = False
+        self.remove_fcbd = False
+        self.remove_publisher = False
 
         # Comic Vine settings
         self.use_series_start_as_volume = False
@@ -187,6 +196,9 @@ class ComicTaggerSettings:
         self.rename_issue_number_padding = 3
         self.rename_use_smart_string_cleanup = True
         self.rename_extension_based_on_archive = True
+        self.rename_dir = ""
+        self.rename_move_dir = False
+        self.rename_strict = False
 
         # Auto-tag stickies
         self.save_on_low_confidence = False
@@ -197,12 +209,15 @@ class ComicTaggerSettings:
         self.wait_and_retry_on_rate_limit = False
 
         self.config = configparser.RawConfigParser()
-        self.folder = ComicTaggerSettings.get_settings_folder()
+        if folder:
+            ComicTaggerSettings.folder = pathlib.Path(folder)
+        else:
+            ComicTaggerSettings.folder = ComicTaggerSettings.get_settings_folder()
 
-        if not os.path.exists(self.folder):
-            os.makedirs(self.folder)
+        if not os.path.exists(ComicTaggerSettings.folder):
+            os.makedirs(ComicTaggerSettings.folder)
 
-        self.settings_file = os.path.join(self.folder, "settings")
+        self.settings_file = os.path.join(ComicTaggerSettings.folder, "settings")
 
         # if config file doesn't exist, write one out
         if not os.path.exists(self.settings_file):
@@ -230,7 +245,7 @@ class ComicTaggerSettings:
 
     def reset(self):
         os.unlink(self.settings_file)
-        self.__init__()
+        self.__init__(ComicTaggerSettings.folder)
 
     def load(self):
         def readline_generator(f):
@@ -239,7 +254,7 @@ class ComicTaggerSettings:
                 yield line
                 line = f.readline()
 
-        with open(self.settings_file, "r") as f:
+        with open(self.settings_file, "r", encoding="utf-8") as f:
             self.config.read_file(readline_generator(f))
 
         self.rar_exe_path = self.config.get("settings", "rar_exe_path")
@@ -278,8 +293,14 @@ class ComicTaggerSettings:
         if self.config.has_option("identifier", "id_publisher_filter"):
             self.id_publisher_filter = self.config.get("identifier", "id_publisher_filter")
 
-        if self.config.has_option("filenameparser", "parse_scan_info"):
-            self.parse_scan_info = self.config.getboolean("filenameparser", "parse_scan_info")
+        if self.config.has_option("filenameparser", "complicated_parser"):
+            self.complicated_parser = self.config.getboolean("filenameparser", "complicated_parser")
+        if self.config.has_option("filenameparser", "remove_c2c"):
+            self.remove_c2c = self.config.getboolean("filenameparser", "remove_c2c")
+        if self.config.has_option("filenameparser", "remove_fcbd"):
+            self.remove_fcbd = self.config.getboolean("filenameparser", "remove_fcbd")
+        if self.config.has_option("filenameparser", "remove_publisher"):
+            self.remove_publisher = self.config.getboolean("filenameparser", "remove_publisher")
 
         if self.config.has_option("dialogflags", "ask_about_cbi_in_rar"):
             self.ask_about_cbi_in_rar = self.config.getboolean("dialogflags", "ask_about_cbi_in_rar")
@@ -344,6 +365,12 @@ class ComicTaggerSettings:
             self.rename_extension_based_on_archive = self.config.getboolean(
                 "rename", "rename_extension_based_on_archive"
             )
+        if self.config.has_option("rename", "rename_dir"):
+            self.rename_dir = self.config.get("rename", "rename_dir")
+        if self.config.has_option("rename", "rename_move_dir"):
+            self.rename_move_dir = self.config.getboolean("rename", "rename_move_dir")
+        if self.config.has_option("rename", "rename_strict"):
+            self.rename_strict = self.config.getboolean("rename", "rename_strict")
 
         if self.config.has_option("autotag", "save_on_low_confidence"):
             self.save_on_low_confidence = self.config.getboolean("autotag", "save_on_low_confidence")
@@ -404,7 +431,10 @@ class ComicTaggerSettings:
         if not self.config.has_section("filenameparser"):
             self.config.add_section("filenameparser")
 
-        self.config.set("filenameparser", "parse_scan_info", self.parse_scan_info)
+        self.config.set("filenameparser", "complicated_parser", self.complicated_parser)
+        self.config.set("filenameparser", "remove_c2c", self.remove_c2c)
+        self.config.set("filenameparser", "remove_fcbd", self.remove_fcbd)
+        self.config.set("filenameparser", "remove_publisher", self.remove_publisher)
 
         if not self.config.has_section("comicvine"):
             self.config.add_section("comicvine")
@@ -441,6 +471,9 @@ class ComicTaggerSettings:
         self.config.set("rename", "rename_issue_number_padding", self.rename_issue_number_padding)
         self.config.set("rename", "rename_use_smart_string_cleanup", self.rename_use_smart_string_cleanup)
         self.config.set("rename", "rename_extension_based_on_archive", self.rename_extension_based_on_archive)
+        self.config.set("rename", "rename_dir", self.rename_dir)
+        self.config.set("rename", "rename_move_dir", self.rename_move_dir)
+        self.config.set("rename", "rename_strict", self.rename_strict)
 
         if not self.config.has_section("autotag"):
             self.config.add_section("autotag")
@@ -451,5 +484,5 @@ class ComicTaggerSettings:
         self.config.set("autotag", "remove_archive_after_successful_match", self.remove_archive_after_successful_match)
         self.config.set("autotag", "wait_and_retry_on_rate_limit", self.wait_and_retry_on_rate_limit)
 
-        with open(self.settings_file, "w") as configfile:
+        with open(self.settings_file, "w", encoding="utf-8") as configfile:
             self.config.write(configfile)

comictaggerlib/settingswindow.py

@@ -21,8 +21,10 @@ import platform
 from PyQt5 import QtCore, QtGui, QtWidgets, uic
 
 from comicapi import utils
+from comicapi.genericmetadata import md_test
 from comictaggerlib.comicvinecacher import ComicVineCacher
 from comictaggerlib.comicvinetalker import ComicVineTalker
+from comictaggerlib.filerenamer import FileRenamer
 from comictaggerlib.imagefetcher import ImageFetcher
 from comictaggerlib.settings import ComicTaggerSettings
 
@@ -55,6 +57,67 @@ macRarHelp = """
                 """
 
 
+template_tooltip = """
+<pre>The template for the new filename. Uses python format strings https://docs.python.org/3/library/string.html#format-string-syntax
+Accepts the following variables:
+{is_empty}       (boolean)
+{tag_origin}     (string)
+{series}        (string)
+{issue}     (string)
+{title}     (string)
+{publisher}     (string)
+{month}     (integer)
+{year}      (integer)
+{day}       (integer)
+{issue_count}    (integer)
+{volume}        (integer)
+{genre}     (string)
+{language}      (string)
+{comments}      (string)
+{volume_count}   (integer)
+{critical_rating}    (string)
+{country}       (string)
+{alternate_series}   (string)
+{alternate_number}   (string)
+{alternate_count}    (integer)
+{imprint}       (string)
+{notes}     (string)
+{web_link}       (string)
+{format}        (string)
+{manga}     (string)
+{black_and_white} (boolean)
+{page_count}     (integer)
+{maturity_rating}    (string)
+{community_rating}     (string)
+{story_arc}      (string)
+{series_group}   (string)
+{scan_info}      (string)
+{characters}    (string)
+{teams}     (string)
+{locations}     (string)
+{credits}       (list of dict({&apos;role&apos;: string, &apos;person&apos;: string, &apos;primary&apos;: boolean}))
+{tags}      (list of str)
+{pages}     (list of dict({&apos;Image&apos;: string(int), &apos;Type&apos;: string}))
+
+CoMet-only items:
+{price}     (float)
+{is_version_of}   (string)
+{rights}        (string)
+{identifier}    (string)
+{last_mark}  (string)
+{cover_image}    (string)
+
+Examples:
+
+{series} {issue} ({year})
+Spider-Geddon 1 (2018)
+
+{series} #{issue} - {title}
+Spider-Geddon #1 - New Players; Check In
+</pre>
+"""
+
+
 class SettingsWindow(QtWidgets.QDialog):
     def __init__(self, parent, settings):
         super().__init__(parent)
@@ -105,15 +168,46 @@ class SettingsWindow(QtWidgets.QDialog):
         validator = QtGui.QIntValidator(0, 99, self)
         self.leNameLengthDeltaThresh.setValidator(validator)
 
+        self.leRenameTemplate.setToolTip(template_tooltip)
         self.settings_to_form()
+        self.rename_error = None
+        self.rename_test()
 
         self.btnBrowseRar.clicked.connect(self.select_rar)
         self.btnClearCache.clicked.connect(self.clear_cache)
         self.btnResetSettings.clicked.connect(self.reset_settings)
         self.btnTestKey.clicked.connect(self.test_api_key)
+        self.btnTemplateHelp.clicked.connect(self.show_template_help)
+        self.leRenameTemplate.textEdited.connect(self.rename__test)
+        self.cbxMoveFiles.clicked.connect(self.rename_test)
+        self.cbxRenameStrict.clicked.connect(self.rename_test)
+        self.leDirectory.textEdited.connect(self.rename_test)
+        self.cbxComplicatedParser.clicked.connect(self.switch_parser)
+
+    def rename_test(self):
+        self.rename__test(self.leRenameTemplate.text())
+
+    def rename__test(self, template):
+        fr = FileRenamer(md_test, platform="universal" if self.cbxRenameStrict.isChecked() else "auto")
+        fr.move = self.cbxMoveFiles.isChecked()
+        fr.set_template(template)
+        fr.set_issue_zero_padding(int(self.leIssueNumPadding.text()))
+        fr.set_smart_cleanup(self.cbxSmartCleanup.isChecked())
+        try:
+            self.lblRenameTest.setText(fr.determine_name(".cbz"))
+            self.rename_error = None
+        except Exception as e:
+            self.rename_error = e
+            self.lblRenameTest.setText(str(e))
+
+    def switch_parser(self):
+        complicated = self.cbxComplicatedParser.isChecked()
+
+        self.cbxRemoveC2C.setEnabled(complicated)
+        self.cbxRemoveFCBD.setEnabled(complicated)
+        self.cbxRemovePublisher.setEnabled(complicated)
 
     def settings_to_form(self):
-
         # Copy values from settings to form
         self.leRarExePath.setText(self.settings.rar_exe_path)
         self.leNameLengthDeltaThresh.setText(str(self.settings.id_length_delta_thresh))
@@ -122,8 +216,11 @@ class SettingsWindow(QtWidgets.QDialog):
         if self.settings.check_for_new_version:
             self.cbxCheckForNewVersion.setCheckState(QtCore.Qt.CheckState.Checked)
 
-        if self.settings.parse_scan_info:
-            self.cbxParseScanInfo.setCheckState(QtCore.Qt.CheckState.Checked)
+        self.cbxComplicatedParser.setChecked(self.settings.complicated_parser)
+        self.cbxRemoveC2C.setChecked(self.settings.remove_c2c)
+        self.cbxRemoveFCBD.setChecked(self.settings.remove_fcbd)
+        self.cbxRemovePublisher.setChecked(self.settings.remove_publisher)
+        self.switch_parser()
 
         if self.settings.use_series_start_as_volume:
             self.cbxUseSeriesStartAsVolume.setCheckState(QtCore.Qt.CheckState.Checked)
@@ -166,8 +263,26 @@ class SettingsWindow(QtWidgets.QDialog):
             self.cbxSmartCleanup.setCheckState(QtCore.Qt.CheckState.Checked)
         if self.settings.rename_extension_based_on_archive:
             self.cbxChangeExtension.setCheckState(QtCore.Qt.CheckState.Checked)
+        if self.settings.rename_move_dir:
+            self.cbxMoveFiles.setCheckState(QtCore.Qt.CheckState.Checked)
+        self.leDirectory.setText(self.settings.rename_dir)
+        if self.settings.rename_strict:
+            self.cbxRenameStrict.setCheckState(QtCore.Qt.CheckState.Checked)
 
     def accept(self):
+        self.rename_test()
+        if self.rename_error is not None:
+            QtWidgets.QMessageBox.critical(
+                self,
+                "Invalid format string!",
+                "Your rename template is invalid!"
+                f"<br/><br/>{self.rename_error}<br/><br/>"
+                "Please consult the template help in the "
+                "settings and the documentation on the format at "
+                "<a href='https://docs.python.org/3/library/string.html#format-string-syntax'>"
+                "https://docs.python.org/3/library/string.html#format-string-syntax</a>",
+            )
+            return
 
         # Copy values from form to settings and save
         self.settings.rar_exe_path = str(self.leRarExePath.text())
@@ -187,7 +302,10 @@ class SettingsWindow(QtWidgets.QDialog):
         self.settings.id_length_delta_thresh = int(self.leNameLengthDeltaThresh.text())
         self.settings.id_publisher_filter = str(self.tePublisherFilter.toPlainText())
 
-        self.settings.parse_scan_info = self.cbxParseScanInfo.isChecked()
+        self.settings.complicated_parser = self.cbxComplicatedParser.isChecked()
+        self.settings.remove_c2c = self.cbxRemoveC2C.isChecked()
+        self.settings.remove_fcbd = self.cbxRemoveFCBD.isChecked()
+        self.settings.remove_publisher = self.cbxRemovePublisher.isChecked()
 
         self.settings.use_series_start_as_volume = self.cbxUseSeriesStartAsVolume.isChecked()
         self.settings.clear_form_before_populating_from_cv = self.cbxClearFormBeforePopulating.isChecked()
@@ -213,6 +331,10 @@ class SettingsWindow(QtWidgets.QDialog):
         self.settings.rename_issue_number_padding = int(self.leIssueNumPadding.text())
         self.settings.rename_use_smart_string_cleanup = self.cbxSmartCleanup.isChecked()
         self.settings.rename_extension_based_on_archive = self.cbxChangeExtension.isChecked()
+        self.settings.rename_move_dir = self.cbxMoveFiles.isChecked()
+        self.settings.rename_dir = self.leDirectory.text()
+
+        self.settings.rename_strict = self.cbxRenameStrict.isChecked()
 
         self.settings.save()
         QtWidgets.QDialog.accept(self)
@@ -262,3 +384,15 @@ class SettingsWindow(QtWidgets.QDialog):
 
     def show_rename_tab(self):
         self.tabWidget.setCurrentIndex(5)
+
+    def show_template_help(self):
+        template_help_win = TemplateHelpWindow(self)
+        template_help_win.setModal(False)
+        template_help_win.show()
+
+
+class TemplateHelpWindow(QtWidgets.QDialog):
+    def __init__(self, parent):
+        super(TemplateHelpWindow, self).__init__(parent)
+
+        uic.loadUi(ComicTaggerSettings.get_ui_file("TemplateHelp.ui"), self)

comictaggerlib/taggerwindow.py

@@ -82,12 +82,12 @@ class TaggerWindow(QtWidgets.QMainWindow):
         alive = socket.waitForConnected(3000)
         if alive:
             print(f"Another application with key [{settings.install_id}] is already running")
-            logger.info(f"Another application with key [{settings.install_id}] is already running")
+            logger.info("Another application with key [%s] is already running", settings.install_id)
             # send file list to other instance
             if file_list:
                 socket.write(pickle.dumps(file_list))
                 if not socket.waitForBytesWritten(3000):
-                    print(socket.errorString())
+                    logger.error(socket.errorString())
             socket.disconnectFromServer()
             sys.exit()
         else:
@@ -360,6 +360,12 @@ Have fun!
         self.actionApplyCBLTransform.setStatusTip("Modify tags specifically for CBL format")
         self.actionApplyCBLTransform.triggered.connect(self.apply_cbl_transform)
 
+        self.actionReCalcPageDims.setShortcut("Ctrl+R")
+        self.actionReCalcPageDims.setStatusTip(
+            "Trigger re-calculating image size, height and width for all pages on the next save"
+        )
+        self.actionReCalcPageDims.triggered.connect(self.recalc_page_dimensions)
+
         self.actionClearEntryForm.setShortcut("Ctrl+Shift+C")
         self.actionClearEntryForm.setStatusTip("Clear all the data on the screen")
         self.actionClearEntryForm.triggered.connect(self.clear_form)
@@ -478,8 +484,7 @@ Please choose options below, and select OK.
                                 os.unlink(ca.path)
 
                         else:
-                            # last export failed, so remove the zip, if it
-                            # exists
+                            # last export failed, so remove the zip, if it exists
                             failed_list.append(ca.path)
                             if os.path.lexists(export_name):
                                 os.remove(export_name)
@@ -552,7 +557,12 @@ Please choose options below, and select OK.
 
     def actual_load_current_archive(self):
         if self.metadata.is_empty:
-            self.metadata = self.comic_archive.metadata_from_filename(self.settings.parse_scan_info)
+            self.metadata = self.comic_archive.metadata_from_filename(
+                self.settings.complicated_parser,
+                self.settings.remove_c2c,
+                self.settings.remove_fcbd,
+                remove_publisher=self.settings.remove_publisher,
+            )
         if len(self.metadata.pages) == 0:
             self.metadata.set_default_page_list(self.comic_archive.get_number_of_pages())
 
@@ -589,6 +599,7 @@ Please choose options below, and select OK.
         self.actionAutoIdentify.setEnabled(False)
         self.actionRename.setEnabled(False)
         self.actionApplyCBLTransform.setEnabled(False)
+        self.actionReCalcPageDims.setEnabled(False)
 
         # now, selectively re-enable
         if self.comic_archive is not None:
@@ -600,6 +611,7 @@ Please choose options below, and select OK.
             self.actionAutoTag.setEnabled(True)
             self.actionRename.setEnabled(True)
             self.actionApplyCBLTransform.setEnabled(True)
+            self.actionReCalcPageDims.setEnabled(True)
             self.actionRepackage.setEnabled(True)
             self.actionRemoveAuto.setEnabled(True)
             self.actionRemoveCRTags.setEnabled(True)
@@ -752,6 +764,11 @@ Please choose options below, and select OK.
         assign_text(self.teTeams, md.teams)
         assign_text(self.teLocations, md.locations)
 
+        try:
+            self.dsbCommunityRating.setValue(float(md.community_rating))
+        except:
+            self.dsbCommunityRating.setValue(0.0)
+
         if md.format is not None and md.format != "":
             i = self.cbFormat.findText(md.format)
             if i == -1:
@@ -867,6 +884,10 @@ Please choose options below, and select OK.
         md.notes = self.teNotes.toPlainText()
         md.maturity_rating = self.cbMaturityRating.currentText()
 
+        md.community_rating = utils.xlate(self.dsbCommunityRating.cleanText())
+        if md.community_rating == "0.0":
+            md.community_rating = None
+
         md.story_arc = self.leStoryArc.text()
         md.scan_info = self.leScanInfo.text()
         md.series_group = self.leSeriesGroup.text()
@@ -912,7 +933,12 @@ Please choose options below, and select OK.
         if self.comic_archive is not None:
             # copy the form onto metadata object
             self.form_to_metadata()
-            new_metadata = self.comic_archive.metadata_from_filename(self.settings.parse_scan_info)
+            new_metadata = self.comic_archive.metadata_from_filename(
+                self.settings.complicated_parser,
+                self.settings.remove_c2c,
+                self.settings.remove_fcbd,
+                remove_publisher=self.settings.remove_publisher,
+            )
             if new_metadata is not None:
                 self.metadata.overlay(new_metadata)
                 self.metadata_to_form()
@@ -1075,7 +1101,6 @@ Please choose options below, and select OK.
 
     def update_credit_colors(self):
         # !!!ATB qt5 porting TODO
-        # return
         inactive_color = QtGui.QColor(255, 170, 150)
         active_palette = self.leSeries.palette()
         active_color = active_palette.color(QtGui.QPalette.ColorRole.Base)
@@ -1166,6 +1191,7 @@ Please choose options below, and select OK.
             self.teLocations,
             self.cbMaturityRating,
             self.cbFormat,
+            self.dsbCommunityRating,
         ]
 
         if self.save_data_style == MetaDataStyle.CIX:
@@ -1363,6 +1389,7 @@ Please choose options below, and select OK.
         self.cbMaturityRating.addItem("PG", "")
         self.cbMaturityRating.addItem("Kids to Adults", "")
         self.cbMaturityRating.addItem("Teen", "")
+        self.cbMaturityRating.addItem("M", "")
         self.cbMaturityRating.addItem("MA15+", "")
         self.cbMaturityRating.addItem("Mature 17+", "")
         self.cbMaturityRating.addItem("R18+", "")
@@ -1637,7 +1664,12 @@ Please choose options below, and select OK.
         # read in metadata, and parse file name if not there
         md = ca.read_metadata(self.save_data_style)
         if md.is_empty:
-            md = ca.metadata_from_filename(self.settings.parse_scan_info)
+            md = ca.metadata_from_filename(
+                self.settings.complicated_parser,
+                self.settings.remove_c2c,
+                self.settings.remove_fcbd,
+                remove_publisher=self.settings.remove_publisher,
+            )
             if dlg.ignore_leading_digits_in_filename and md.series is not None:
                 # remove all leading numbers
                 md.series = re.sub(r"([\d.]*)(.*)", "\\2", md.series)
@@ -1829,7 +1861,9 @@ Please choose options below, and select OK to Auto-Tag.
 
             match_results.multiple_matches.extend(match_results.low_confidence_matches)
             if reply == QtWidgets.QMessageBox.StandardButton.Yes:
-                matchdlg = AutoTagMatchWindow(self, match_results.multiple_matches, style, self.actual_issue_data_fetch)
+                matchdlg = AutoTagMatchWindow(
+                    self, match_results.multiple_matches, style, self.actual_issue_data_fetch, self.settings
+                )
                 matchdlg.setModal(True)
                 matchdlg.exec()
                 self.fileSelectionList.update_selected_rows()
@@ -1928,6 +1962,18 @@ Please choose options below, and select OK to Auto-Tag.
         self.metadata = CBLTransformer(self.metadata, self.settings).apply()
         self.metadata_to_form()
 
+    def recalc_page_dimensions(self):
+        QtWidgets.QApplication.setOverrideCursor(QtGui.QCursor(QtCore.Qt.CursorShape.WaitCursor))
+        for p in self.metadata.pages:
+            if "ImageSize" in p:
+                del p["ImageSize"]
+            if "ImageHeight" in p:
+                del p["ImageHeight"]
+            if "ImageWidth" in p:
+                del p["ImageWidth"]
+        self.set_dirty_flag()
+        QtWidgets.QApplication.restoreOverrideCursor()
+
     def rename_archive(self):
         ca_list = self.fileSelectionList.get_selected_archive_list()
 
@@ -2037,8 +2083,8 @@ Please choose options below, and select OK to Auto-Tag.
                 # the top
                 win32gui.SetWindowPos(hwnd, win32con.HWND_TOPMOST, x, y, w, h, 0)
                 win32gui.SetWindowPos(hwnd, win32con.HWND_NOTOPMOST, x, y, w, h, 0)
-            except Exception as e:
-                print("Whoops", e)
+            except Exception:
+                logger.exception("Fail to bring window to top")
         elif platform.system() == "Darwin":
             self.raise_()
             self.showNormal()

comictaggerlib/ui/TemplateHelp.ui

@@ -0,0 +1,135 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<ui version="4.0">
+ <class>Dialog</class>
+ <widget class="QDialog" name="Dialog">
+  <property name="geometry">
+   <rect>
+    <x>0</x>
+    <y>0</y>
+    <width>702</width>
+    <height>452</height>
+   </rect>
+  </property>
+  <property name="windowTitle">
+   <string>Template Help</string>
+  </property>
+  <property name="sizeGripEnabled">
+   <bool>true</bool>
+  </property>
+  <layout class="QVBoxLayout" name="verticalLayout">
+   <property name="spacing">
+    <number>0</number>
+   </property>
+   <property name="leftMargin">
+    <number>2</number>
+   </property>
+   <property name="topMargin">
+    <number>2</number>
+   </property>
+   <item>
+    <widget class="QTextBrowser" name="textEdit">
+     <property name="readOnly">
+      <bool>true</bool>
+     </property>
+     <property name="html">
+      <string>&lt;html&gt;
+	&lt;head&gt;
+     &lt;style&gt;
+table {
+  font-family: arial, sans-serif;
+  border-collapse: collapse;
+  width: 100%;
+}
+
+td, th {
+  border: 1px solid #dddddd;
+  text-align: left;
+  padding: 8px;
+}
+
+tr:nth-child(even) {
+  background-color: #dddddd;
+}
+&lt;/style&gt;
+     &lt;/head&gt;
+	&lt;body&gt;
+		&lt;h1 style="text-align: center"&gt;Template help&lt;/h1&gt;
+		&lt;p&gt;The template uses Python format strings, in the simplest use it replaces the field (e.g. {issue}) with the value for that particular comic (e.g. 1) for advanced formatting please reference the
+
+			&lt;a href="https://docs.python.org/3/library/string.html#format-string-syntax"&gt;Python 3 documentation&lt;/a&gt;&lt;/p&gt;
+		Accepts the following variables:
+&lt;table&gt;
+     &lt;tr&gt;
+          &lt;th&gt;Tag name&lt;/th&gt;
+          &lt;th&gt;Type&lt;/th&gt;
+     &lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{is_empty}&lt;/td&gt;&lt;td&gt;boolean&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{tag_origin}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{series}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{issue}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{title}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{publisher}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{month}&lt;/td&gt;&lt;td&gt;integer&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{year}&lt;/td&gt;&lt;td&gt;integer&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{day}&lt;/td&gt;&lt;td&gt;integer&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{issue_count}&lt;/td&gt;&lt;td&gt;integer&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{volume}&lt;/td&gt;&lt;td&gt;integer&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{genre}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{language}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{comments}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{volume_count}&lt;/td&gt;&lt;td&gt;integer&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{critical_rating}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{country}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{alternate_series}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{alternate_number}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{alternate_count}&lt;/td&gt;&lt;td&gt;integer&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{imprint}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{notes}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{web_link}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{format}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{manga}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{black_and_white}&lt;/td&gt;&lt;td&gt;boolean&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{page_count}&lt;/td&gt;&lt;td&gt;integer&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{maturity_rating}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{community_rating}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{story_arc}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{series_group}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{scan_info}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{characters}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{teams}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{locations}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{credits}&lt;/td&gt;&lt;td&gt;list of dict({&apos;role&apos;: string, &apos;person&apos;: string, &apos;primary&apos;: boolean})&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{tags}&lt;/td&gt;&lt;td&gt;list of str&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{pages}&lt;/td&gt;&lt;td&gt;list of dict({&apos;Image&apos;: string(int), &apos;Type&apos;: string})&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{price}&lt;/td&gt;&lt;td&gt;float&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{is_version_of}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{rights}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{identifier}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{last_mark}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+     &lt;tr&gt;&lt;td&gt;{cover_image}&lt;/td&gt;&lt;td&gt;string&lt;/td&gt;&lt;/tr&gt;
+&lt;/table&gt;
+&lt;pre&gt;
+Examples:
+
+{series} {issue} ({year})
+Spider-Geddon 1 (2018)
+
+{series} #{issue} - {title}
+Spider-Geddon #1 - New Players; Check In
+
+&lt;/pre&gt;
+	&lt;/body&gt;
+&lt;/html&gt;</string></property>
+     <property name="textInteractionFlags">
+      <set>Qt::TextBrowserInteraction</set>
+     </property>
+     <property name="openExternalLinks">
+      <bool>true</bool>
+     </property>
+    </widget>
+   </item>
+  </layout>
+ </widget>
+ <resources/>
+ <connections/>
+</ui>

comictaggerlib/ui/autotagprogresswindow.ui

@@ -21,7 +21,7 @@
   </property>
   <layout class="QGridLayout" name="gridLayout">
    <item row="0" column="1">
-    <widget class="QWidget" name="archiveCoverContainer">
+    <widget class="QWidget" name="archiveCoverContainer" native="true">
      <property name="sizePolicy">
       <sizepolicy hsizetype="Fixed" vsizetype="Expanding">
        <horstretch>0</horstretch>
@@ -43,7 +43,7 @@
     </widget>
    </item>
    <item row="0" column="4">
-    <widget class="QWidget" name="testCoverContainer">
+    <widget class="QWidget" name="testCoverContainer" native="true">
      <property name="minimumSize">
       <size>
        <width>110</width>

comictaggerlib/ui/coverimagewidget.ui

@@ -14,15 +14,24 @@
    <string>Form</string>
   </property>
   <layout class="QGridLayout" name="gridLayout">
+   <property name="leftMargin">
+    <number>0</number>
+   </property>
+   <property name="topMargin">
+    <number>0</number>
+   </property>
+   <property name="rightMargin">
+    <number>0</number>
+   </property>
+   <property name="bottomMargin">
+    <number>0</number>
+   </property>
    <property name="horizontalSpacing">
     <number>0</number>
    </property>
    <property name="verticalSpacing">
     <number>4</number>
    </property>
-   <property name="margin">
-    <number>0</number>
-   </property>
    <item row="1" column="0">
     <layout class="QHBoxLayout" name="horizontalLayout">
      <property name="spacing">

comictaggerlib/ui/exportwindow.ui

@@ -10,7 +10,7 @@
     <x>0</x>
     <y>0</y>
     <width>533</width>
-    <height>202</height>
+    <height>231</height>
    </rect>
   </property>
   <property name="sizePolicy">

comictaggerlib/ui/fileselectionlist.ui

@@ -28,12 +28,12 @@
      <property name="textElideMode">
       <enum>Qt::ElideMiddle</enum>
      </property>
-     <attribute name="horizontalHeaderDefaultSectionSize">
-      <number>61</number>
-     </attribute>
      <attribute name="horizontalHeaderMinimumSectionSize">
       <number>36</number>
      </attribute>
+     <attribute name="horizontalHeaderDefaultSectionSize">
+      <number>61</number>
+     </attribute>
      <attribute name="horizontalHeaderStretchLastSection">
       <bool>false</bool>
      </attribute>
@@ -45,7 +45,7 @@
        <string>File Name</string>
       </property>
       <property name="textAlignment">
-       <set>AlignHCenter|AlignVCenter|AlignCenter</set>
+       <set>AlignCenter</set>
       </property>
      </column>
      <column>
@@ -56,7 +56,7 @@
        <string>Has ComicRack Tags</string>
       </property>
       <property name="textAlignment">
-       <set>AlignHCenter|AlignVCenter|AlignCenter</set>
+       <set>AlignCenter</set>
       </property>
      </column>
      <column>
@@ -67,7 +67,7 @@
        <string>Has ComicBookLover Tags</string>
       </property>
       <property name="textAlignment">
-       <set>AlignHCenter|AlignVCenter|AlignCenter</set>
+       <set>AlignCenter</set>
       </property>
      </column>
      <column>
@@ -78,7 +78,7 @@
        <string>Archive Type</string>
       </property>
       <property name="textAlignment">
-       <set>AlignHCenter|AlignVCenter|AlignCenter</set>
+       <set>AlignCenter</set>
       </property>
      </column>
      <column>
@@ -89,7 +89,7 @@
        <string>Read-Only</string>
       </property>
       <property name="textAlignment">
-       <set>AlignHCenter|AlignVCenter|AlignCenter</set>
+       <set>AlignCenter</set>
       </property>
      </column>
      <column>
@@ -100,7 +100,7 @@
        <string>File Location</string>
       </property>
       <property name="textAlignment">
-       <set>AlignHCenter|AlignVCenter|AlignCenter</set>
+       <set>AlignCenter</set>
       </property>
      </column>
     </widget>

comictaggerlib/ui/issueselectionwindow.ui

@@ -75,7 +75,7 @@
             <string>Title</string>
            </property>
            <property name="textAlignment">
-            <set>AlignHCenter|AlignVCenter|AlignCenter</set>
+            <set>AlignCenter</set>
            </property>
           </column>
          </widget>

comictaggerlib/ui/pagelisteditor.ui

@@ -87,25 +87,38 @@
    <item>
     <layout class="QVBoxLayout" name="verticalLayout_2">
      <item>
-      <layout class="QFormLayout" name="formLayout">
+      <layout class="QGridLayout" name="gridLayout_1">
        <item row="0" column="0">
-        <widget class="QLabel" name="label_2">
+        <widget class="QLabel" name="lblPageType">
          <property name="text">
           <string>Page Type:</string>
          </property>
+         <property name="alignment">
+          <set>Qt::AlignRight</set>
+         </property>
         </widget>
        </item>
        <item row="0" column="1">
-        <widget class="QComboBox" name="comboBox"/>
+        <widget class="QComboBox" name="cbPageType"/>
+       </item>
+       <item row="0" column="2">
+        <widget class="QCheckBox" name="chkDoublePage">
+         <property name="text">
+          <string>&amp;Double Page?</string>
+         </property>
+        </widget>
        </item>
       </layout>
      </item>
      <item>
       <layout class="QFormLayout" name="formLayout_2">
        <item row="0" column="0">
-        <widget class="QLabel" name="label_3">
+        <widget class="QLabel" name="lblBookmark">
          <property name="text">
-          <string>Bookmark:</string>
+          <string>Book&amp;mark:</string>
+         </property>
+         <property name="buddy">
+          <cstring>leBookmark</cstring>
          </property>
         </widget>
        </item>

comictaggerlib/ui/progresswindow.ui

@@ -28,11 +28,12 @@
      </item>
      <item>
       <widget class="QTextEdit" name="textEdit">
-        <property name="font">
+       <property name="font">
         <font>
          <family>Courier</family>
         </font>
-       </property>      <property name="readOnly">
+       </property>
+       <property name="readOnly">
         <bool>true</bool>
        </property>
       </widget>

comictaggerlib/ui/qtutils.py

@@ -2,13 +2,14 @@
 
 import io
 import logging
+import traceback
 
 from comictaggerlib.settings import ComicTaggerSettings
 
 logger = logging.getLogger(__name__)
 
 try:
-    from PyQt5 import QtGui
+    from PyQt5 import QtGui, QtWidgets
 
     qt_available = True
 except ImportError:
@@ -74,3 +75,10 @@ if qt_available:
         if not success:
             img.load(ComicTaggerSettings.get_graphic("nocover.png"))
         return img
+
+    def qt_error(msg: str, e: Exception = None):
+        trace = ""
+        if e:
+            trace = "\n".join(traceback.format_exception(type(e), e, e.__traceback__))
+
+        QtWidgets.QMessageBox.critical(QtWidgets.QMainWindow(), "Error", msg + trace)

comictaggerlib/ui/settingswindow.ui

@@ -28,7 +28,7 @@
         </sizepolicy>
        </property>
        <property name="currentIndex">
-        <number>1</number>
+        <number>0</number>
        </property>
        <widget class="QWidget" name="tab">
         <attribute name="title">
@@ -229,19 +229,55 @@
         <attribute name="title">
          <string>Filename Parser</string>
         </attribute>
-        <widget class="QCheckBox" name="cbxParseScanInfo">
-         <property name="geometry">
-          <rect>
-           <x>30</x>
-           <y>30</y>
-           <width>421</width>
-           <height>25</height>
-          </rect>
-         </property>
-         <property name="text">
-          <string>Parse Scan Info From Filename (Experimental)</string>
-         </property>
-        </widget>
+        <layout class="QVBoxLayout" name="verticalLayout_6">
+         <item>
+          <widget class="QGroupBox" name="groupBox_2">
+           <layout class="QVBoxLayout" name="verticalLayout_7">
+            <item>
+             <widget class="QCheckBox" name="cbxComplicatedParser">
+              <property name="text">
+               <string>Use &quot;Complicated&quot; Parser</string>
+              </property>
+             </widget>
+            </item>
+            <item>
+             <widget class="QCheckBox" name="cbxRemoveC2C">
+              <property name="text">
+               <string>Remove 'C2C' from Scan Info</string>
+              </property>
+             </widget>
+            </item>
+            <item>
+             <widget class="QCheckBox" name="cbxRemoveFCBD">
+              <property name="text">
+               <string>Remove 'FCBD' from Scan Info</string>
+              </property>
+             </widget>
+            </item>
+            <item>
+             <widget class="QCheckBox" name="cbxRemovePublisher">
+              <property name="text">
+               <string>Remove Publisher from filename</string>
+              </property>
+             </widget>
+            </item>
+           </layout>
+          </widget>
+         </item>
+         <item>
+          <spacer name="verticalSpacer_4">
+           <property name="orientation">
+            <enum>Qt::Vertical</enum>
+           </property>
+           <property name="sizeHint" stdset="0">
+            <size>
+             <width>20</width>
+             <height>40</height>
+            </size>
+           </property>
+          </spacer>
+         </item>
+        </layout>
        </widget>
        <widget class="QWidget" name="tab_3">
         <attribute name="title">
@@ -547,40 +583,43 @@
             <enum>QFormLayout::AllNonFixedFieldsGrow</enum>
            </property>
            <item row="1" column="0">
-            <widget class="QLabel" name="label">
+            <widget class="QLabel" name="lblTemplate">
              <property name="text">
               <string>Template:</string>
              </property>
             </widget>
            </item>
            <item row="1" column="1">
-            <widget class="QLineEdit" name="leRenameTemplate">
-             <property name="toolTip">
-              <string>&lt;html&gt;&lt;head/&gt;&lt;body&gt;&lt;p&gt;The template for the new filename. Accepts the following variables:&lt;/p&gt;&lt;p&gt;%series%&lt;br/&gt;%issue%&lt;br/&gt;%volume%&lt;br/&gt;%issuecount%&lt;br/&gt;%year%&lt;br/&gt;%month%&lt;br/&gt;%month_name%&lt;br/&gt;%publisher%&lt;br/&gt;%title%&lt;br/&gt;
-%genre%&lt;br/&gt;
-%language_code%&lt;br/&gt;
-%criticalrating%&lt;br/&gt;
-%alternateseries%&lt;br/&gt;
-%alternatenumber%&lt;br/&gt;
-%alternatecount%&lt;br/&gt;
-%imprint%&lt;br/&gt;
-%format%&lt;br/&gt;
-%maturityrating%&lt;br/&gt;
-%storyarc%&lt;br/&gt;
-%seriesgroup%&lt;br/&gt;
-%scaninfo%
-&lt;/p&gt;&lt;p&gt;Examples:&lt;/p&gt;&lt;p&gt;&lt;span style=&quot; font-style:italic;&quot;&gt;%series% %issue% (%year%)&lt;/span&gt;&lt;br/&gt;&lt;span style=&quot; font-style:italic;&quot;&gt;%series% #%issue% - %title%&lt;/span&gt;&lt;/p&gt;&lt;/body&gt;&lt;/html&gt;</string>
+            <widget class="QLineEdit" name="leRenameTemplate"/>
+           </item>
+           <item row="2" column="0">
+            <widget class="QPushButton" name="btnTemplateHelp">
+             <property name="text">
+              <string>Template Help</string>
              </property>
             </widget>
            </item>
-           <item row="2" column="0">
-            <widget class="QLabel" name="label_6">
+           <item row="2" column="1">
+            <widget class="QLabel" name="lblRenameTest">
+             <property name="text">
+              <string/>
+             </property>
+             <property name="textFormat">
+              <enum>Qt::PlainText</enum>
+             </property>
+             <property name="textInteractionFlags">
+              <set>Qt::LinksAccessibleByMouse|Qt::TextSelectableByKeyboard|Qt::TextSelectableByMouse</set>
+             </property>
+            </widget>
+           </item>
+           <item row="3" column="0">
+            <widget class="QLabel" name="lblPadding">
              <property name="text">
               <string>Issue # Zero Padding</string>
              </property>
             </widget>
            </item>
-           <item row="2" column="1">
+           <item row="3" column="1">
             <widget class="QLineEdit" name="leIssueNumPadding">
              <property name="sizePolicy">
               <sizepolicy hsizetype="Maximum" vsizetype="Fixed">
@@ -599,7 +638,7 @@
              </property>
             </widget>
            </item>
-           <item row="3" column="0" colspan="2">
+           <item row="4" column="0" colspan="2">
             <widget class="QCheckBox" name="cbxSmartCleanup">
              <property name="toolTip">
               <string>&lt;html&gt;&lt;head/&gt;&lt;body&gt;&lt;p&gt;&lt;span style=&quot; font-weight:600;&quot;&gt;&amp;quot;Smart Text Cleanup&amp;quot; &lt;/span&gt;will attempt to clean up the new filename if there are missing fields from the template.  For example, removing empty braces, repeated spaces and dashes, and more.  Experimental feature.&lt;/p&gt;&lt;/body&gt;&lt;/html&gt;</string>
@@ -609,13 +648,44 @@
              </property>
             </widget>
            </item>
-           <item row="4" column="0" colspan="2">
+           <item row="5" column="0" colspan="2">
             <widget class="QCheckBox" name="cbxChangeExtension">
              <property name="text">
               <string>Change Extension Based On Archive Type</string>
              </property>
             </widget>
            </item>
+           <item row="7" column="0">
+            <widget class="QCheckBox" name="cbxMoveFiles">
+             <property name="toolTip">
+              <string>If checked moves files to specified folder</string>
+             </property>
+             <property name="text">
+              <string>Move files when renaming</string>
+             </property>
+            </widget>
+           </item>
+           <item row="9" column="0">
+            <widget class="QLabel" name="lblDirectory">
+             <property name="text">
+              <string>Destination Directory:</string>
+             </property>
+            </widget>
+           </item>
+           <item row="9" column="1">
+            <widget class="QLineEdit" name="leDirectory"/>
+           </item>
+           <item row="8" column="0">
+            <widget class="QCheckBox" name="cbxRenameStrict">
+             <property name="toolTip">
+              <string>If checked will ensure reserved characters and filenames are removed for all Operating Systems.
+By default only removes restricted characters and filenames for the current Operating System.</string>
+             </property>
+             <property name="text">
+              <string>Strict renaming</string>
+             </property>
+            </widget>
+           </item>
           </layout>
          </item>
         </layout>

comictaggerlib/ui/taggerwindow.ui

@@ -108,7 +108,16 @@
              <enum>QFrame::Sunken</enum>
             </property>
             <layout class="QVBoxLayout" name="verticalLayout_4">
-             <property name="margin">
+             <property name="leftMargin">
+              <number>6</number>
+             </property>
+             <property name="topMargin">
+              <number>6</number>
+             </property>
+             <property name="rightMargin">
+              <number>6</number>
+             </property>
+             <property name="bottomMargin">
               <number>6</number>
              </property>
              <item>
@@ -807,7 +816,7 @@
                       <string>Primary</string>
                      </property>
                      <property name="textAlignment">
-                      <set>AlignHCenter|AlignVCenter|AlignCenter</set>
+                      <set>AlignCenter</set>
                      </property>
                     </column>
                     <column>
@@ -929,31 +938,31 @@
                  </widget>
                 </item>
                 <item row="2" column="1">
-                  <layout class="QGridLayout" name="gridLayout_7">
-                   <item row="0" column="0">
-                    <widget class="QLineEdit" name="leWebLink">
-                     <property name="acceptDrops">
-                      <bool>false</bool>
-                     </property>
-                    </widget>
-                   </item>
-                   <item row="0" column="1">
-                    <widget class="QPushButton" name="btnOpenWebLink">
-                     <property name="sizePolicy">
-                      <sizepolicy hsizetype="Maximum" vsizetype="Fixed">
-                       <horstretch>0</horstretch>
-                       <verstretch>0</verstretch>
-                      </sizepolicy>
-                     </property>
-                     <property name="maximumSize">
-                      <size>
-                       <width>40</width>
-                       <height>16777215</height>
-                      </size>
-                     </property>
-                    </widget>
-                   </item>
-                  </layout>
+                 <layout class="QGridLayout" name="gridLayout_7">
+                  <item row="0" column="0">
+                   <widget class="QLineEdit" name="leWebLink">
+                    <property name="acceptDrops">
+                     <bool>false</bool>
+                    </property>
+                   </widget>
+                  </item>
+                  <item row="0" column="1">
+                   <widget class="QPushButton" name="btnOpenWebLink">
+                    <property name="sizePolicy">
+                     <sizepolicy hsizetype="Maximum" vsizetype="Fixed">
+                      <horstretch>0</horstretch>
+                      <verstretch>0</verstretch>
+                     </sizepolicy>
+                    </property>
+                    <property name="maximumSize">
+                     <size>
+                      <width>40</width>
+                      <height>16777215</height>
+                     </size>
+                    </property>
+                   </widget>
+                  </item>
+                 </layout>
                 </item>
                 <item row="3" column="0">
                  <widget class="QLabel" name="userRatingLabel">
@@ -981,6 +990,35 @@
                   </property>
                  </widget>
                 </item>
+                <item row="4" column="0">
+                 <widget class="QLabel" name="lblCommunityRating">
+                  <property name="text">
+                   <string>Community Rating</string>
+                  </property>
+                 </widget>
+                </item>
+                <item row="4" column="1">
+                 <widget class="QDoubleSpinBox" name="dsbCommunityRating">
+                  <property name="minimumSize">
+                   <size>
+                    <width>80</width>
+                    <height>0</height>
+                   </size>
+                  </property>
+                  <property name="decimals">
+                   <number>1</number>
+                  </property>
+                  <property name="minimum">
+                   <double>0.000000000000000</double>
+                  </property>
+                  <property name="maximum">
+                   <double>5.000000000000000</double>
+                  </property>
+                  <property name="singleStep">
+                   <double>0.100000000000000</double>
+                  </property>
+                 </widget>
+                </item>
                </layout>
               </item>
              </layout>
@@ -1144,7 +1182,7 @@
      <x>0</x>
      <y>0</y>
      <width>1096</width>
-     <height>22</height>
+     <height>21</height>
     </rect>
    </property>
    <widget class="QMenu" name="menuComicTagger">
@@ -1198,6 +1236,7 @@
     <addaction name="actionAutoIdentify"/>
     <addaction name="separator"/>
     <addaction name="actionApplyCBLTransform"/>
+    <addaction name="actionReCalcPageDims"/>
    </widget>
    <widget class="QMenu" name="menuWindow">
     <property name="title">
@@ -1372,6 +1411,11 @@
     <string>Apply CBL Transform</string>
    </property>
   </action>
+  <action name="actionReCalcPageDims">
+   <property name="text">
+    <string>Re-Calculate Page Dimensions</string>
+   </property>
+  </action>
   <action name="actionLoadFolder">
    <property name="text">
     <string>Open Folder</string>

comictaggerlib/ui/volumeselectionwindow.ui

@@ -6,7 +6,7 @@
    <rect>
     <x>0</x>
     <y>0</y>
-    <width>849</width>
+    <width>893</width>
     <height>476</height>
    </rect>
   </property>
@@ -86,7 +86,7 @@
             <string>Year</string>
            </property>
            <property name="textAlignment">
-            <set>AlignHCenter|AlignVCenter|AlignCenter</set>
+            <set>AlignCenter</set>
            </property>
           </column>
           <column>
@@ -94,7 +94,7 @@
             <string>Issues</string>
            </property>
            <property name="textAlignment">
-            <set>AlignHCenter|AlignVCenter|AlignCenter</set>
+            <set>AlignCenter</set>
            </property>
           </column>
           <column>
@@ -148,16 +148,16 @@
            </property>
           </widget>
          </item>
-        <item>
-         <widget class="QCheckBox" name="cbxFilter">
-          <property name="text">
-           <string>Filter Publishers</string>
-          </property>
-          <property name="toolTip">
-           <string>Filter the publishers based on the publisher filter.</string>
-          </property>
-         </widget>
-        </item>
+         <item>
+          <widget class="QCheckBox" name="cbxFilter">
+           <property name="toolTip">
+            <string>Filter the publishers based on the publisher filter.</string>
+           </property>
+           <property name="text">
+            <string>Filter Publishers</string>
+           </property>
+          </widget>
+         </item>
          <item>
           <widget class="QDialogButtonBox" name="buttonBox">
            <property name="orientation">

comictaggerlib/volumeselectionwindow.py

@@ -130,6 +130,11 @@ class VolumeSelectionWindow(QtWidgets.QDialog):
         self.immediate_autoselect = autoselect
         self.cover_index_list = cover_index_list
         self.cv_search_results = None
+        self.ii = None
+        self.iddialog = None
+        self.id_thread = None
+        self.progdialog = None
+        self.search_thread = None
 
         self.use_filter = self.settings.always_use_publisher_filter
 
@@ -343,7 +348,7 @@ class VolumeSelectionWindow(QtWidgets.QDialog):
 
         # pre sort the data - so that we can put exact matches first afterwards
         # compare as str incase extra chars ie. '1976?'
-        # - missing (none) values being converted to 'None' - consistant with prior behaviour in v1.2.3
+        # - missing (none) values being converted to 'None' - consistent with prior behaviour in v1.2.3
         # sort by start_year if set
         if self.settings.sort_series_by_year:
             try:

localefix.py

@@ -23,9 +23,9 @@ def _lang_code_mac():
     # - The macOS underlying API:
     #   https://developer.apple.com/documentation/foundation/nsuserdefaults.
 
-    LANG_DETECT_COMMAND = "defaults read -g AppleLocale"
+    lang_detect_command = "defaults read -g AppleLocale"
 
-    status, output = subprocess.getstatusoutput(LANG_DETECT_COMMAND)
+    status, output = subprocess.getstatusoutput(lang_detect_command)
     if status == 0:
         # Command was successful.
         lang_code = output

requirements.txt

@@ -2,5 +2,7 @@ beautifulsoup4 >= 4.1
 natsort>=8.1.0
 pillow>=4.3.0
 requests==2.*
+pathvalidate
 pycountry
 py7zr
+text2digits

requirements_dev.txt

@@ -4,4 +4,6 @@ setuptools_scm[toml]>=3.4
 wheel
 black>=22
 flake8==4.*
+flake8-encodings
 isort>=5.10
+pytest==7.*

scripts/name_fixer.py

@@ -92,8 +92,6 @@ def main():
     else:
         xform_list = default_xform_list
 
-    # pprint( xform_list, indent=4)
-
     filelist = utils.get_recursive_filelist(parsed_args.paths)
 
     # first find all comics

setup.py

@@ -24,7 +24,7 @@ def read(fname):
     str
         File contents.
     """
-    with open(os.path.join(os.path.dirname(__file__), fname)) as f:
+    with open(os.path.join(os.path.dirname(__file__), fname), encoding="utf-8") as f:
         return f.read()
 
 
@@ -48,7 +48,7 @@ setup(
     name="comictagger",
     install_requires=install_requires,
     extras_require=extras_require,
-    python_requires=">=3",
+    python_requires=">=3.9",
     description="A cross-platform GUI/CLI app for writing metadata to comic archives",
     author="ComicTagger team",
     author_email="comictagger@gmail.com",
@@ -68,8 +68,7 @@ setup(
         "License :: OSI Approved :: Apache Software License",
         "Natural Language :: English",
         "Operating System :: OS Independent",
-        "Programming Language :: Python :: 3.5",
-        "Programming Language :: Python :: 3.6",
+        "Programming Language :: Python :: 3.9",
         "Topic :: Utilities",
         "Topic :: Other/Nonlisted Topic",
         "Topic :: Multimedia :: Graphics",

tests/filenames.py

@@ -0,0 +1,714 @@
+fnames = [
+    (
+        "batman 3 title (DC).cbz",
+        "honorific and publisher in series",
+        {
+            "issue": "3",
+            "series": "batman",
+            "title": "title",
+            "publisher": "DC",
+            "volume": "",
+            "year": "",
+            "remainder": "",
+            "issue_count": "",
+            "alternate": "",
+        },
+    ),
+    (
+        "batman 3 title DC.cbz",
+        "honorific and publisher in series",
+        {
+            "issue": "3",
+            "series": "batman",
+            "title": "title DC",
+            "publisher": "DC",
+            "volume": "",
+            "year": "",
+            "remainder": "",
+            "issue_count": "",
+            "alternate": "",
+        },
+    ),
+    (
+        "ms. Marvel 3.cbz",
+        "honorific and publisher in series",
+        {
+            "issue": "3",
+            "series": "ms. Marvel",
+            "title": "",
+            "publisher": "Marvel",
+            "volume": "",
+            "year": "",
+            "remainder": "",
+            "issue_count": "",
+            "alternate": "",
+        },
+    ),
+    (
+        "january jones 2.cbz",
+        "month in series",
+        {
+            "issue": "2",
+            "series": "january jones",
+            "title": "",
+            "volume": "",
+            "year": "",
+            "remainder": "",
+            "issue_count": "",
+            "alternate": "",
+        },
+    ),
+    (
+        "52.cbz",
+        "issue number only",
+        {
+            "issue": "52",
+            "series": "",
+            "title": "",
+            "volume": "",
+            "year": "",
+            "remainder": "",
+            "issue_count": "",
+            "alternate": "",
+        },
+    ),
+    (
+        "52 Monster_Island_v1_2__repaired__c2c.cbz",
+        "leading alternate",
+        {
+            "issue": "2",
+            "series": "Monster Island",
+            "title": "",
+            "volume": "1",
+            "year": "",
+            "remainder": "repaired",
+            "issue_count": "",
+            "alternate": "52",
+            "c2c": True,
+        },
+    ),
+    (
+        "Monster_Island_v1_2__repaired__c2c.cbz",
+        "Example from userguide",
+        {
+            "issue": "2",
+            "series": "Monster Island",
+            "title": "",
+            "volume": "1",
+            "year": "",
+            "remainder": "repaired",
+            "issue_count": "",
+            "c2c": True,
+        },
+    ),
+    (
+        "Monster Island v1 3 (1957) -- The Revenge Of King Klong (noads).cbz",
+        "Example from userguide",
+        {
+            "issue": "3",
+            "series": "Monster Island",
+            "title": "",
+            "volume": "1",
+            "year": "1957",
+            "remainder": "The Revenge Of King Klong (noads)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Foobar-Man Annual 121 - The Wrath of Foobar-Man, Part 1 of 2.cbz",
+        "Example from userguide",
+        {
+            "issue": "121",
+            "series": "Foobar-Man Annual",
+            "title": "The Wrath of Foobar-Man, Part 1 of 2",
+            "volume": "",
+            "year": "",
+            "remainder": "",
+            "issue_count": "",
+            "annual": True,
+        },
+    ),
+    (
+        "Plastic Man v1 002 (1942).cbz",
+        "Example from userguide",
+        {
+            "issue": "2",
+            "series": "Plastic Man",
+            "title": "",
+            "volume": "1",
+            "year": "1942",
+            "remainder": "",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Blue Beetle 02.cbr",
+        "Example from userguide",
+        {
+            "issue": "2",
+            "series": "Blue Beetle",
+            "title": "",
+            "volume": "",
+            "year": "",
+            "remainder": "",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Monster Island vol. 2 #2.cbz",
+        "Example from userguide",
+        {
+            "issue": "2",
+            "series": "Monster Island",
+            "title": "",
+            "volume": "2",
+            "year": "",
+            "remainder": "",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Crazy Weird Comics 2 (of 2) (1969).rar",
+        "Example from userguide",
+        {
+            "issue": "2",
+            "series": "Crazy Weird Comics",
+            "title": "",
+            "volume": "",
+            "year": "1969",
+            "remainder": "",
+            "issue_count": "2",
+        },
+    ),
+    (
+        "Super Strange Yarns (1957) #92 (1969).cbz",
+        "Example from userguide",
+        {
+            "issue": "92",
+            "series": "Super Strange Yarns",
+            "title": "",
+            "volume": "1957",
+            "year": "1969",
+            "remainder": "",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Action Spy Tales v1965 #3.cbr",
+        "Example from userguide",
+        {
+            "issue": "3",
+            "series": "Action Spy Tales",
+            "title": "",
+            "volume": "1965",
+            "year": "",
+            "remainder": "",
+            "issue_count": "",
+        },
+    ),
+    (
+        " X-Men-V1-067.cbr",
+        "hyphen separated with hyphen in series",  # only parses corretly because v1 designates the volume
+        {
+            "issue": "67",
+            "series": "X-Men",
+            "title": "",
+            "volume": "1",
+            "year": "",
+            "remainder": "",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Amazing Spider-Man 078.BEY (2022) (Digital) (Zone-Empire).cbr",
+        "number issue with extra",
+        {
+            "issue": "78.BEY",
+            "series": "Amazing Spider-Man",
+            "title": "",
+            "volume": "",
+            "year": "2022",
+            "remainder": "(Digital) (Zone-Empire)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Angel Wings 02 - Black Widow (2015) (Scanlation) (phillywilly).cbr",
+        "title after issue",
+        {
+            "issue": "2",
+            "series": "Angel Wings",
+            "title": "Black Widow",
+            "volume": "",
+            "year": "2015",
+            "remainder": "(Scanlation) (phillywilly)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Angel Wings #02 - Black Widow (2015) (Scanlation) (phillywilly).cbr",
+        "title after #issue",
+        {
+            "issue": "2",
+            "series": "Angel Wings",
+            "title": "Black Widow",
+            "volume": "",
+            "year": "2015",
+            "remainder": "(Scanlation) (phillywilly)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Aquaman - Green Arrow - Deep Target 01 (of 07) (2021) (digital) (Son of Ultron-Empire).cbr",
+        "issue count",
+        {
+            "issue": "1",
+            "series": "Aquaman - Green Arrow - Deep Target",
+            "title": "",
+            "volume": "",
+            "year": "2021",
+            "issue_count": "7",
+            "remainder": "(digital) (Son of Ultron-Empire)",
+        },
+    ),
+    (
+        "Aquaman 80th Anniversary 100-Page Super Spectacular (2021) 001 (2021) (Digital) (BlackManta-Empire).cbz",
+        "numbers in series",
+        {
+            "issue": "1",
+            "series": "Aquaman 80th Anniversary 100-Page Super Spectacular",
+            "title": "",
+            "volume": "2021",
+            "year": "2021",
+            "remainder": "(Digital) (BlackManta-Empire)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Avatar - The Last Airbender - The Legend of Korra (FCBD 2021) (Digital) (mv-DCP).cbr",
+        "FCBD date",
+        {
+            "issue": "",
+            "series": "Avatar - The Last Airbender - The Legend of Korra",
+            "title": "",
+            "volume": "",
+            "year": "2021",
+            "remainder": "(Digital) (mv-DCP)",
+            "issue_count": "",
+            "fcbd": True,
+        },
+    ),
+    (
+        "Avengers By Brian Michael Bendis v03 (2013) (Digital) (F2) (Kileko-Empire).cbz",
+        "volume without issue",
+        {
+            "issue": "",
+            "series": "Avengers By Brian Michael Bendis",
+            "title": "",
+            "volume": "3",
+            "year": "2013",
+            "remainder": "(Digital) (F2) (Kileko-Empire)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Batman '89 (2021) (Webrip) (The Last Kryptonian-DCP).cbr",
+        "year in title without issue",
+        {
+            "issue": "",
+            "series": "Batman '89",
+            "title": "",
+            "volume": "",
+            "year": "2021",
+            "remainder": "(Webrip) (The Last Kryptonian-DCP)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Batman_-_Superman_020_(2021)_(digital)_(NeverAngel-Empire).cbr",
+        "underscores",
+        {
+            "issue": "20",
+            "series": "Batman - Superman",
+            "title": "",
+            "volume": "",
+            "year": "2021",
+            "remainder": "(digital) (NeverAngel-Empire)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Black Widow 009 (2021) (Digital) (Zone-Empire).cbr",
+        "standard",
+        {
+            "issue": "9",
+            "series": "Black Widow",
+            "title": "",
+            "volume": "",
+            "year": "2021",
+            "remainder": "(Digital) (Zone-Empire)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Blade Runner 2029 006 (2021) (3 covers) (digital) (Son of Ultron-Empire).cbr",
+        "year before issue",
+        {
+            "issue": "6",
+            "series": "Blade Runner 2029",
+            "title": "",
+            "volume": "",
+            "year": "2021",
+            "remainder": "(3 covers) (digital) (Son of Ultron-Empire)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Blade Runner Free Comic Book Day 2021 (2021) (digital-Empire).cbr",
+        "FCBD year and (year)",
+        {
+            "issue": "",
+            "series": "Blade Runner Free Comic Book Day 2021",
+            "title": "",
+            "volume": "",
+            "year": "2021",
+            "remainder": "(digital-Empire)",
+            "issue_count": "",
+            "fcbd": True,
+        },
+    ),
+    (
+        "Bloodshot Book 03 (2020) (digital) (Son of Ultron-Empire).cbr",
+        "book",
+        {
+            "issue": "3",
+            "series": "Bloodshot",
+            "title": "Book 03",
+            "volume": "3",
+            "year": "2020",
+            "remainder": "(digital) (Son of Ultron-Empire)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "book of eli (2020) (digital) (Son of Ultron-Empire).cbr",
+        "book",
+        {
+            "issue": "",
+            "series": "book of eli",
+            "title": "",
+            "volume": "",
+            "year": "2020",
+            "remainder": "(digital) (Son of Ultron-Empire)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Cyberpunk 2077 - You Have My Word 02 (2021) (digital) (Son of Ultron-Empire).cbr",
+        "title",
+        {
+            "issue": "2",
+            "series": "Cyberpunk 2077",
+            "title": "You Have My Word",
+            "volume": "",
+            "year": "2021",
+            "issue_count": "",
+            "remainder": "(digital) (Son of Ultron-Empire)",
+        },
+    ),
+    (
+        "Elephantmen 2259 008 - Simple Truth 03 (of 06) (2021) (digital) (Son of Ultron-Empire).cbr",
+        "volume count",
+        {
+            "issue": "8",
+            "series": "Elephantmen 2259",
+            "title": "Simple Truth",
+            "volume": "3",
+            "year": "2021",
+            "volume_count": "6",
+            "remainder": "(digital) (Son of Ultron-Empire)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Elephantmen 2259 #008 - Simple Truth 03 (of 06) (2021) (digital) (Son of Ultron-Empire).cbr",
+        "volume count",
+        {
+            "issue": "8",
+            "series": "Elephantmen 2259",
+            "title": "Simple Truth",
+            "volume": "3",
+            "year": "2021",
+            "volume_count": "6",
+            "remainder": "(digital) (Son of Ultron-Empire)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Free Comic Book Day - Avengers.Hulk (2021) (2048px) (db).cbz",
+        "'.' in name",
+        {
+            "issue": "",
+            "series": "Free Comic Book Day - Avengers Hulk",
+            "title": "",
+            "volume": "",
+            "year": "2021",
+            "remainder": "(2048px) (db)",
+            "issue_count": "",
+            "fcbd": True,
+        },
+    ),
+    (
+        "Goblin (2021) (digital) (Son of Ultron-Empire).cbr",
+        "no-issue",
+        {
+            "issue": "",
+            "series": "Goblin",
+            "title": "",
+            "volume": "",
+            "year": "2021",
+            "remainder": "(digital) (Son of Ultron-Empire)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Marvel Previews 002 (January 2022) (Digital-Empire).cbr",
+        "(month year)",
+        {
+            "issue": "2",
+            "series": "Marvel Previews",
+            "title": "",
+            "publisher": "Marvel",
+            "volume": "",
+            "year": "2022",
+            "remainder": "(Digital-Empire)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Marvel Two In One V1 090  c2c (Comixbear-DCP).cbr",
+        "volume issue ctc",
+        {
+            "issue": "90",
+            "series": "Marvel Two In One",
+            "title": "",
+            "publisher": "Marvel",
+            "volume": "1",
+            "year": "",
+            "remainder": "(Comixbear-DCP)",
+            "issue_count": "",
+            "c2c": True,
+        },
+    ),
+    (
+        "Marvel Two In One V1 #090  c2c (Comixbear-DCP).cbr",
+        "volume then issue",
+        {
+            "issue": "90",
+            "series": "Marvel Two In One",
+            "title": "",
+            "publisher": "Marvel",
+            "volume": "1",
+            "year": "",
+            "remainder": "(Comixbear-DCP)",
+            "issue_count": "",
+            "c2c": True,
+        },
+    ),
+    (
+        "Star Wars - War of the Bounty Hunters - IG-88 (2021) (Digital) (Kileko-Empire).cbz",
+        "number ends series, no-issue",
+        {
+            "issue": "",
+            "series": "Star Wars - War of the Bounty Hunters - IG-88",
+            "title": "",
+            "volume": "",
+            "year": "2021",
+            "remainder": "(Digital) (Kileko-Empire)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Star Wars - War of the Bounty Hunters - IG-88 #1 (2021) (Digital) (Kileko-Empire).cbz",
+        "number ends series",
+        {
+            "issue": "1",
+            "series": "Star Wars - War of the Bounty Hunters - IG-88",
+            "title": "",
+            "volume": "",
+            "year": "2021",
+            "remainder": "(Digital) (Kileko-Empire)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "The Defenders v1 058 (1978) (digital).cbz",
+        "",
+        {
+            "issue": "58",
+            "series": "The Defenders",
+            "title": "",
+            "volume": "1",
+            "year": "1978",
+            "remainder": "(digital)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "The Defenders v1 Annual 01 (1976) (Digital) (Minutemen-Slayer).cbr",
+        " v in series",
+        {
+            "issue": "1",
+            "series": "The Defenders Annual",
+            "title": "",
+            "volume": "1",
+            "year": "1976",
+            "remainder": "(Digital) (Minutemen-Slayer)",
+            "issue_count": "",
+            "annual": True,
+        },
+    ),
+    (
+        "The Magic Order 2 06 (2022) (Digital) (Zone-Empire)[__913302__].cbz",
+        "ending id",
+        {
+            "issue": "6",
+            "series": "The Magic Order 2",
+            "title": "",
+            "volume": "",
+            "year": "2022",
+            "remainder": "(Digital) (Zone-Empire)[913302]",  # Don't really care about double underscores
+            "issue_count": "",
+        },
+    ),
+    (
+        "Wonder Woman 001 Wonder Woman Day Special Edition (2021) (digital-Empire).cbr",
+        "issue separates title",
+        {
+            "issue": "1",
+            "series": "Wonder Woman",
+            "title": "Wonder Woman Day Special Edition",
+            "volume": "",
+            "year": "2021",
+            "remainder": "(digital-Empire)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Wonder Woman #001 Wonder Woman Day Special Edition (2021) (digital-Empire).cbr",
+        "issue separates title",
+        {
+            "issue": "1",
+            "series": "Wonder Woman",
+            "title": "Wonder Woman Day Special Edition",
+            "volume": "",
+            "year": "2021",
+            "remainder": "(digital-Empire)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Wonder Woman 49 DC Sep-Oct 1951 digital [downsized, lightened, 4 missing story pages restored] (Shadowcat-Empire).cbz",
+        "date-range, no paren, braces",
+        {
+            "issue": "49",
+            "series": "Wonder Woman",
+            "title": "digital",  # Don't have a way to get rid of this
+            "publisher": "DC",
+            "volume": "",
+            "year": "1951",
+            "remainder": "[downsized, lightened, 4 missing story pages restored] (Shadowcat-Empire)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "Wonder Woman #49 DC Sep-Oct 1951 digital [downsized, lightened, 4 missing story pages restored] (Shadowcat-Empire).cbz",
+        "date-range, no paren, braces",
+        {
+            "issue": "49",
+            "series": "Wonder Woman",
+            "title": "digital",  # Don't have a way to get rid of this
+            "publisher": "DC",
+            "volume": "",
+            "year": "1951",
+            "remainder": "[downsized, lightened, 4 missing story pages restored] (Shadowcat-Empire)",
+            "issue_count": "",
+        },
+    ),
+    (
+        "X-Men, 2021-08-04 (#02) (digital) (Glorith-HD).cbz",
+        "full-date, issue in parenthesis",
+        {
+            "issue": "2",
+            "series": "X-Men",
+            "title": "",
+            "volume": "",
+            "year": "2021",
+            "remainder": "(digital) (Glorith-HD)",
+            "issue_count": "",
+        },
+    ),
+]
+
+rnames = [
+    (
+        "{series} #{issue} - {title} ({year}) ({price})",  # price should be none
+        False,
+        "universal",
+        "Cory Doctorow's Futuristic Tales of the Here and Now #001 - Anda's Game (2007).cbz",
+    ),
+    (
+        "{series} #{issue} - {title} ({year})",
+        False,
+        "universal",
+        "Cory Doctorow's Futuristic Tales of the Here and Now #001 - Anda's Game (2007).cbz",
+    ),
+    (
+        "{series}: {title} #{issue} ({year})",
+        False,
+        "universal",
+        "Cory Doctorow's Futuristic Tales of the Here and Now - Anda's Game #001 (2007).cbz",
+    ),
+    (
+        "{series}: {title} #{issue} ({year})",
+        False,
+        "Linux",
+        "Cory Doctorow's Futuristic Tales of the Here and Now: Anda's Game #001 (2007).cbz",
+    ),
+    (
+        "{publisher}/  {series} #{issue} - {title} ({year})",
+        True,
+        "universal",
+        "IDW Publishing/Cory Doctorow's Futuristic Tales of the Here and Now #001 - Anda's Game (2007).cbz",
+    ),
+    (
+        "{publisher}/  {series} #{issue} - {title} ({year})",
+        False,
+        "universal",
+        "Cory Doctorow's Futuristic Tales of the Here and Now #001 - Anda's Game (2007).cbz",
+    ),
+    (
+        r"{publisher}\  {series} #{issue} - {title} ({year})",
+        False,
+        "universal",
+        "Cory Doctorow's Futuristic Tales of the Here and Now #001 - Anda's Game (2007).cbz",
+    ),
+    (
+        "{series} #  {issue} - {title} ({year})",
+        False,
+        "universal",
+        "Cory Doctorow's Futuristic Tales of the Here and Now # 001 - Anda's Game (2007).cbz",
+    ),
+    (
+        "{series} #  {issue} - {locations} ({year})",
+        False,
+        "universal",
+        "Cory Doctorow's Futuristic Tales of the Here and Now # 001 - lonely cottage (2007).cbz",
+    ),
+    (
+        "{series} #{issue} - {title} - {WriteR}, {EDITOR} ({year})",
+        False,
+        "universal",
+        "Cory Doctorow's Futuristic Tales of the Here and Now #001 - Anda's Game - Dara Naraghi, Ted Adams (2007).cbz",
+    ),
+]

tests/test_FilenameParser.py

@@ -0,0 +1,42 @@
+import pytest
+from filenames import fnames
+
+import comicapi.filenameparser
+
+
+@pytest.mark.parametrize("filename,reason,expected", fnames)
+def test_file_name_parser_new(filename, reason, expected):
+    p = comicapi.filenameparser.Parse(
+        comicapi.filenamelexer.Lex(filename).items,
+        first_is_alt=True,
+        remove_c2c=True,
+        remove_fcbd=True,
+        remove_publisher=True,
+    )
+    fp = p.filename_info
+
+    for s in ["archive"]:
+        if s in fp:
+            del fp[s]
+    for s in ["alternate", "publisher", "volume_count"]:
+        if s not in expected:
+            expected[s] = ""
+    for s in ["fcbd", "c2c", "annual"]:
+        if s not in expected:
+            expected[s] = False
+
+    assert fp == expected
+
+
+@pytest.mark.parametrize("filename,reason,expected", fnames)
+def test_file_name_parser(filename, reason, expected):
+    p = comicapi.filenameparser.FileNameParser()
+    p.parse_filename(filename)
+    fp = p.__dict__
+    for s in ["title", "alternate", "publisher", "fcbd", "c2c", "annual", "volume_count"]:
+        if s in expected:
+            del expected[s]
+
+    if fp != expected:
+        pytest.xfail("old parser")
+    assert fp == expected

tests/test_comicarchive.py

@@ -0,0 +1,75 @@
+import shutil
+from os.path import abspath, dirname, join
+
+import pytest
+
+from comicapi.comicarchive import ComicArchive, rar_support
+from comicapi.genericmetadata import GenericMetadata, md_test
+
+thisdir = dirname(abspath(__file__))
+
+
+@pytest.mark.xfail(not rar_support, reason="rar support")
+def test_getPageNameList():
+    ComicArchive.logo_data = b""
+    c = ComicArchive(join(thisdir, "data", "fake_cbr.cbr"))
+    pageNameList = c.get_page_name_list()
+
+    assert pageNameList == [
+        "page0.jpg",
+        "Page1.jpeg",
+        "Page2.png",
+        "Page3.gif",
+        "page4.webp",
+        "page10.jpg",
+    ]
+
+
+def test_set_default_page_list(tmpdir):
+    md = GenericMetadata()
+    md.overlay(md_test)
+    md.pages = []
+    md.set_default_page_list(len(md_test.pages))
+
+    assert isinstance(md.pages[0]["Image"], int)
+
+
+def test_page_type_read():
+    c_path = join(thisdir, "data", "Cory Doctorow's Futuristic Tales of the Here and Now #001 - Anda's Game (2007).cbz")
+    c = ComicArchive(str(c_path))
+    md = c.read_cix()
+
+    assert isinstance(md.pages[0]["Type"], str)
+
+
+def test_save_cix(tmpdir):
+    comic_path = tmpdir.mkdir("cbz").join(
+        "Cory Doctorow's Futuristic Tales of the Here and Now #001 - Anda's Game (2007).cbz"
+    )
+    c_path = join(thisdir, "data", "Cory Doctorow's Futuristic Tales of the Here and Now #001 - Anda's Game (2007).cbz")
+    shutil.copy(c_path, comic_path)
+
+    c = ComicArchive(str(comic_path))
+    md = c.read_cix()
+    md.set_default_page_list(c.get_number_of_pages())
+
+    assert c.write_cix(md)
+
+    md = c.read_cix()
+
+
+def test_page_type_save(tmpdir):
+    comic_path = tmpdir.mkdir("cbz").join(
+        "Cory Doctorow's Futuristic Tales of the Here and Now #001 - Anda's Game (2007).cbz"
+    )
+    c_path = join(thisdir, "data", "Cory Doctorow's Futuristic Tales of the Here and Now #001 - Anda's Game (2007).cbz")
+    shutil.copy(c_path, comic_path)
+
+    c = ComicArchive(str(comic_path))
+    md = c.read_cix()
+    t = md.pages[0]
+    t["Type"] = ""
+
+    assert c.write_cix(md)
+
+    md = c.read_cix()

tests/test_rename.py

@@ -0,0 +1,15 @@
+import pathlib
+
+import pytest
+from filenames import rnames
+
+from comicapi.genericmetadata import md_test
+from comictaggerlib.filerenamer import FileRenamer
+
+
+@pytest.mark.parametrize("template, move, platform, expected", rnames)
+def test_rename(template, platform, move, expected):
+    fr = FileRenamer(md_test, platform=platform)
+    fr.move = move
+    fr.set_template(template)
+    assert str(pathlib.PureWindowsPath(fr.determine_name(".cbz"))) == str(pathlib.PureWindowsPath(expected))