Compare commits

..

20 Commits

Author SHA1 Message Date
50c53e14da Add seriesYear attribute 2021-12-15 11:02:50 -08:00
2e01a4db14 Improve file renaming
Moves to Python format strings for renaming, handles directory
structures, moving of files to a destination directory, sanitizes
file paths with pathvalidate and takes a different approach to
smart filename cleanup using the Python string.Formatter class

Moving to Python format strings means we can point to python
documentation for syntax and all we have to do is document the
properties and types that are attached to the GenericMetadata class.

Switching to pathvalidate allows comictagger to more simply handle both
directories and symbols in filenames.

The only changes to the string.Formatter class is:
1. format_field returns
an empty string if the value is none or an empty string regardless of
the format specifier.
2. _vformat drops the previous literal text if the field value
is an empty string and lstrips the following literal text of closing
special characters.
2021-12-15 11:00:01 -08:00
444e67100c Merge pull request #207 from jpcranford/patch-1
Fixed typo
2021-12-15 08:49:15 -08:00
82d054fd05 Fixed typo 2021-12-14 16:52:48 -07:00
f82c024f8d Merge pull request #206 from lordwelch/rarOptionalFix
Fix rarfile import as by default it is optional
2021-12-12 18:49:05 -08:00
da4daa6a8a Fix rarfile import as by default it is optional 2021-12-12 18:46:28 -08:00
6e1e8959c9 Merge pull request #204 from lordwelch/buildSystem
Update build
2021-12-12 18:15:58 -08:00
aedc5bedb4 Update build
Separate dependencies into files and add optional dependencies
Update natsort usage to be compliant with the latest version (#203)
Set PyQt5 to 5.15.3, 5.15.4 has issues with pyinstaller
Add pyproject.toml with setuptools, isort and black configuration
Add optional dependencies (#191)
Update README (#174)
2021-10-23 21:39:58 -07:00
93f5061c8f Add GitHub Actions yaml file (#201)
Upload artifacts this allows easy testing of macOS and Windows binaries
Update unrar-cffi for Python 3.9 wheels
2021-09-29 01:17:04 -07:00
d46e171bd6 Merge pull request #199 from lordwelch/seriesSearch
Improve issue identification
2021-09-26 17:09:54 -07:00
e7fe520660 Improve issue identification
Move title sanitizing code to utils module
Update issue identifier to compare sanitized names
2021-09-26 17:06:30 -07:00
91f288e8f4 Update travis
hold windows to 3.7.9 as unrar-cffi only has windows wheels for 3.7
switch to using builtin python for macOS
remove ssl dlls from comictagger.spec
require pyinstaller=4.3 to allow macOS codesigning
Update python usage
restrict builds to tags and pull requests
2021-09-26 12:51:17 -07:00
d7bd3bb94b Merge pull request #198 from lordwelch/143-regression
Fix regression of #143
2021-09-25 23:01:38 -07:00
9e0b0ac01c Fix regression of #143 2021-09-25 22:59:59 -07:00
03a8d906ea Merge pull request #189 from lordwelch/seriesSearch
Series search
2021-09-21 19:59:26 -07:00
fff28cf6ae Improve searchForSeries
Refactor removearticles to only remove articles
Add normalization on the search string and the series name results

Searching now only compares ASCII a-z and 0-9 and all other characters
are replaced with single space, this is done to both the search string
and the result. This fixes an with names that are separated by a
hyphen (-) in the filename but in the Comic Vine name are separated by a
slash (/) and other similar issues.
2021-08-29 17:35:34 -07:00
9ee95b8d5e Merge pull request #192 from lordwelch/fixes
Fix errors
2021-08-16 17:37:19 -07:00
11bf5a9709 Move to python requests module
Add requests to requirements.txt
Requests is much simpler and fixes all ssl errors.
Comic Vine now requires a unique useragent string
2021-08-11 20:13:53 -07:00
af4b3af14e Cleanup metadata handling
Mainly corrects for consistency in most situations
CoMet is not touched as there is no support in the gui and has an odd requirements on attributes
2021-08-07 21:54:29 -07:00
9bb7fbbc9e Fix errors
Libraries updated and these are no longer needed
2021-08-05 17:21:21 -07:00
33 changed files with 1110 additions and 646 deletions

63
.github/workflows/build.yaml vendored Normal file
View File

@ -0,0 +1,63 @@
name: Build
on:
push:
tags:
- "[0-9]+.[0-9]+.[0-9]+*"
pull_request:
jobs:
build:
permissions:
contents: write
runs-on: ${{ matrix.os }}
strategy:
matrix:
python-version: [3.9]
os: [ubuntu-latest, macos-10.15, windows-latest]
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python3 -m pip install -r requirements_dev.txt
python3 -m pip install -r requirements.txt
for requirement in requirements-*.txt; do
python3 -m pip install -r "$requirement"
done
shell: bash
- name: Install Windows dependencies
run: |
choco install -y zip
if: runner.os == 'Windows'
- name: build
run: |
make pydist
make dist
- name: Archive production artifacts
uses: actions/upload-artifact@v2
if: runner.os != 'Linux' # linux binary currently has a segfault when running on latest fedora
with:
name: "${{ format('ComicTagger-{0}', runner.os) }}"
path: dist/*.zip
- name: Release
uses: softprops/action-gh-release@v1
if: startsWith(github.ref, 'refs/tags/')
with:
prerelease: "${{ contains(github.ref, '-') }}" # alpha-releases should be 1.3.0-alpha.x full releases should be 1.3.0
draft: true
files: dist/*.zip
- name: "Publish distribution 📦 to PyPI"
if: startsWith(github.ref, 'refs/tags/') && runner.os == 'Linux'
uses: pypa/gh-action-pypi-publish@master
with:
password: ${{ secrets.PYPI_API_TOKEN }}
packages_dir: piprelease

167
.gitignore vendored
View File

@ -1,12 +1,157 @@
/.idea/
/nbproject/
/dist
*.pyc
/.vscode
venv
*.o
*.a
*.so
build/
# generated by setuptools_scm
ctversion.py
.eggs
# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion
*.iml
## Directory-based project format:
.idea/
### Other editors
.*.swp
nbproject/
.vscode
comictaggerlib/_version.py
*.exe
*.zip
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
.pybuilder/
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# pytype static type analyzer
.pytype/
# Cython debug symbols
cython_debug/

View File

@ -1,30 +1,37 @@
language: python
# Only build tags
if: type = pull_request OR tag IS present
branches:
only:
- develop
- /^\d+\.\d+\.\d+.*$/
env:
global:
- PYTHON=python
- PIP=pip
- PYTHON=python3
- PIP=pip3
- SETUPTOOLS_SCM_PRETEND_VERSION=$TRAVIS_TAG
- MAKE=make
matrix:
include:
- os: linux
- os: osx
language: generic
osx_image: xcode8.3
env: PYTHON=python3 PIP=pip3 MACOSX_DEPLOYMENT_TARGET=10.11
python: 3.8
- name: "Python: 3.7"
os: osx
language: shell
python: 3.7
env: PYTHON=python3 PIP="python3 -m pip"
cache:
- directories:
- $HOME/Library/Caches/pip
- os: windows
language: bash
env: PATH=/C/Python37:/C/Python37/Scripts:$PATH MAKE=mingw32-make
env: PATH=/C/Python37:/C/Python37/Scripts:$PATH MAKE=mingw32-make PIP=pip PYTHON=python
before_install:
- if [ "$TRAVIS_OS_NAME" = "windows" ]; then choco install -y python mingw zip; fi
- if [ "$TRAVIS_OS_NAME" = "osx" ]; then brew upgrade python3 ; fi
- if [ "$TRAVIS_OS_NAME" = "windows" ]; then choco install -y python --version 3.7.9; choco install -y mingw zip; fi
install:
- $PIP install --upgrade setuptools
- $PIP install -r requirements.txt
- $PIP install -r requirements_dev.txt
- $PIP install -r requirements-GUI.txt
- $PIP install -r requirements-CBR.txt
script:
- if [ "$TRAVIS_OS_NAME" != "linux" ]; then $MAKE dist ; fi
@ -49,4 +56,4 @@ deploy:
skip_cleanup: true
on:
tags: true
condition: $TRAVIS_OS_NAME = "linux"
condition: $TRAVIS_OS_NAME = "linux"

View File

@ -1,5 +1,6 @@
PIP ?= pip
VERSION_STR := $(shell python setup.py --version)
PIP ?= pip3
PYTHON ?= python3
VERSION_STR := $(shell $(PYTHON) setup.py --version)
ifeq ($(OS),Windows_NT)
OS_VERSION=win-$(PROCESSOR_ARCHITECTURE)
@ -28,19 +29,19 @@ clean:
$(MAKE) -C mac clean
rm -rf build
rm -rf comictaggerlib/ui/__pycache__
rm comitaggerlib/ctversion.py
rm comictaggerlib/ctversion.py
pydist:
make clean
mkdir -p piprelease
rm -f comictagger-$(VERSION_STR).zip
python setup.py sdist --formats=zip #,gztar
mv dist/comictagger-$(VERSION_STR).zip piprelease
$(PYTHON) setup.py sdist --formats=gztar
mv dist/comictagger-$(VERSION_STR).tar.gz piprelease
rm -rf comictagger.egg-info dist
upload:
python setup.py register
python setup.py sdist --formats=zip upload
$(PYTHON) setup.py register
$(PYTHON) setup.py sdist --formats=gztar upload
dist:
$(PIP) install .

View File

@ -37,12 +37,14 @@ Just unzip the archive in any folder and run, no additional installation steps a
A pip package is provided, you can install it with:
```
$ pip install comictagger
$ pip3 install comictagger[GUI]
```
### From source
1. ensure you have a recent version of python3 and setuptools installed
2. clone this repository `git clone https://github.com/comictagger/comictagger.git`
3. `pip install -r requirements.txt`
4. `python comictagger.py`
1. Ensure you have a recent version of python3 installed
2. Clone this repository `git clone https://github.com/comictagger/comictagger.git`
3. `pip3 install -r requirements_dev.txt`
4. Optionally install the GUI `pip3 install -r requirements-GUI.txt`
5. Optionally install CBR support `pip3 install -r requirements-CBR.txt`
6. `python3 comictagger.py`

View File

@ -24,9 +24,13 @@ import platform
import time
import io
from natsort import natsorted
import natsort
from PyPDF2 import PdfFileReader
from unrar.cffi import rarfile
try:
from unrar.cffi import rarfile
except:
pass
try:
import Image
pil_available = True
@ -627,7 +631,10 @@ class ComicArchive:
return zipfile.is_zipfile(self.path)
def rarTest(self):
return rarfile.is_rarfile(self.path)
try:
return rarfile.is_rarfile(self.path)
except:
return False
def isZip(self):
return self.archive_type == self.ArchiveType.Zip
@ -809,13 +816,9 @@ class ComicArchive:
# about case-sensitivity!
if sort_list:
def keyfunc(k):
# hack to account for some weird scanner ID pages
# basename=os.path.split(k)[1]
# if basename < '0':
# k = os.path.join(os.path.split(k)[0], "z" + basename)
return k.lower()
files = natsorted(files, key=keyfunc, signed=False)
files = natsort.natsorted(files, alg=natsort.ns.IC | natsort.ns.I)
# make a sub-list of image files
self.page_list = []

View File

@ -24,39 +24,34 @@ from . import utils
class ComicBookInfo:
def metadataFromString(self, string):
class Default(dict):
def __missing__(self, key):
return None
cbi_container = json.loads(str(string, 'utf-8'))
metadata = GenericMetadata()
cbi = cbi_container['ComicBookInfo/1.0']
cbi = Default(cbi_container['ComicBookInfo/1.0'])
# helper func
# If item is not in CBI, return None
def xlate(cbi_entry):
if cbi_entry in cbi:
return cbi[cbi_entry]
else:
return None
metadata.series = utils.xlate(cbi['series'])
metadata.title = utils.xlate(cbi['title'])
metadata.issue = utils.xlate(cbi['issue'])
metadata.publisher = utils.xlate(cbi['publisher'])
metadata.month = utils.xlate(cbi['publicationMonth'], True)
metadata.year = utils.xlate(cbi['publicationYear'], True)
metadata.seriesYear = utils.xlate('seriesPublicationYear', True)
metadata.issueCount = utils.xlate(cbi['numberOfIssues'], True)
metadata.comments = utils.xlate(cbi['comments'])
metadata.genre = utils.xlate(cbi['genre'])
metadata.volume = utils.xlate(cbi['volume'], True)
metadata.volumeCount = utils.xlate(cbi['numberOfVolumes'], True)
metadata.language = utils.xlate(cbi['language'])
metadata.country = utils.xlate(cbi['country'])
metadata.criticalRating = utils.xlate(cbi['rating'])
metadata.series = xlate('series')
metadata.title = xlate('title')
metadata.issue = xlate('issue')
metadata.publisher = xlate('publisher')
metadata.month = xlate('publicationMonth')
metadata.year = xlate('publicationYear')
metadata.issueCount = xlate('numberOfIssues')
metadata.comments = xlate('comments')
metadata.credits = xlate('credits')
metadata.genre = xlate('genre')
metadata.volume = xlate('volume')
metadata.volumeCount = xlate('numberOfVolumes')
metadata.language = xlate('language')
metadata.country = xlate('country')
metadata.criticalRating = xlate('rating')
metadata.tags = xlate('tags')
metadata.credits = cbi['credits']
metadata.tags = cbi['tags']
# make sure credits and tags are at least empty lists and not None
if metadata.credits is None:
@ -103,33 +98,24 @@ class ComicBookInfo:
# helper func
def assign(cbi_entry, md_entry):
if md_entry is not None:
if md_entry is not None or isinstance(md_entry, str) and md_entry != "":
cbi[cbi_entry] = md_entry
# helper func
def toInt(s):
i = None
if type(s) in [str, str, int]:
try:
i = int(s)
except ValueError:
pass
return i
assign('series', metadata.series)
assign('title', metadata.title)
assign('issue', metadata.issue)
assign('publisher', metadata.publisher)
assign('publicationMonth', toInt(metadata.month))
assign('publicationYear', toInt(metadata.year))
assign('numberOfIssues', toInt(metadata.issueCount))
assign('comments', metadata.comments)
assign('genre', metadata.genre)
assign('volume', toInt(metadata.volume))
assign('numberOfVolumes', toInt(metadata.volumeCount))
assign('language', utils.getLanguageFromISO(metadata.language))
assign('country', metadata.country)
assign('rating', metadata.criticalRating)
assign('series', utils.xlate(metadata.series))
assign('title', utils.xlate(metadata.title))
assign('issue', utils.xlate(metadata.issue))
assign('publisher', utils.xlate(metadata.publisher))
assign('publicationMonth', utils.xlate(metadata.month, True))
assign('publicationYear', utils.xlate(metadata.year, True))
assign('seriesPublicationYear', utils.xlate(metadata.seriesYear, True))
assign('numberOfIssues', utils.xlate(metadata.issueCount, True))
assign('comments', utils.xlate(metadata.comments))
assign('genre', utils.xlate(metadata.genre))
assign('volume', utils.xlate(metadata.volume, True))
assign('numberOfVolumes', utils.xlate(metadata.volumeCount, True))
assign('language', utils.xlate(utils.getLanguageFromISO(metadata.language)))
assign('country', utils.xlate(metadata.country))
assign('rating', utils.xlate(metadata.criticalRating))
assign('credits', metadata.credits)
assign('tags', metadata.tags)

View File

@ -20,6 +20,7 @@ import xml.etree.ElementTree as ET
#import zipfile
from .genericmetadata import GenericMetadata
from .issuestring import IssueString
from . import utils
@ -103,6 +104,7 @@ class ComicInfoXml:
assign('Year', md.year)
assign('Month', md.month)
assign('Day', md.day)
assign('SeriesYear', md.seriesYear)
# need to specially process the credits, since they are structured
# differently than CIX
@ -206,48 +208,45 @@ class ComicInfoXml:
raise 1
return None
metadata = GenericMetadata()
md = metadata
# Helper function
def xlate(tag):
node = root.find(tag)
if node is not None:
return node.text
else:
def get(name):
tag = root.find(name)
if tag is None:
return None
return tag.text
md.series = xlate('Series')
md.title = xlate('Title')
md.issue = xlate('Number')
md.issueCount = xlate('Count')
md.volume = xlate('Volume')
md.alternateSeries = xlate('AlternateSeries')
md.alternateNumber = xlate('AlternateNumber')
md.alternateCount = xlate('AlternateCount')
md.comments = xlate('Summary')
md.notes = xlate('Notes')
md.year = xlate('Year')
md.month = xlate('Month')
md.day = xlate('Day')
md.publisher = xlate('Publisher')
md.imprint = xlate('Imprint')
md.genre = xlate('Genre')
md.webLink = xlate('Web')
md.language = xlate('LanguageISO')
md.format = xlate('Format')
md.manga = xlate('Manga')
md.characters = xlate('Characters')
md.teams = xlate('Teams')
md.locations = xlate('Locations')
md.pageCount = xlate('PageCount')
md.scanInfo = xlate('ScanInformation')
md.storyArc = xlate('StoryArc')
md.seriesGroup = xlate('SeriesGroup')
md.maturityRating = xlate('AgeRating')
md = GenericMetadata()
tmp = xlate('BlackAndWhite')
md.blackAndWhite = False
md.series = utils.xlate(get('Series'))
md.title = utils.xlate(get('Title'))
md.issue = IssueString(utils.xlate(get('Number'))).asString()
md.issueCount = utils.xlate(get('Count'), True)
md.volume = utils.xlate(get('Volume'), True)
md.alternateSeries = utils.xlate(get('AlternateSeries'))
md.alternateNumber = IssueString(utils.xlate(get('AlternateNumber'))).asString()
md.alternateCount = utils.xlate(get('AlternateCount'), True)
md.comments = utils.xlate(get('Summary'))
md.notes = utils.xlate(get('Notes'))
md.year = utils.xlate(get('Year'), True)
md.month = utils.xlate(get('Month'), True)
md.day = utils.xlate(get('Day'), True)
md.seriesYear = utils.xlate(get('SeriesYear'), True)
md.publisher = utils.xlate(get('Publisher'))
md.imprint = utils.xlate(get('Imprint'))
md.genre = utils.xlate(get('Genre'))
md.webLink = utils.xlate(get('Web'))
md.language = utils.xlate(get('LanguageISO'))
md.format = utils.xlate(get('Format'))
md.manga = utils.xlate(get('Manga'))
md.characters = utils.xlate(get('Characters'))
md.teams = utils.xlate(get('Teams'))
md.locations = utils.xlate(get('Locations'))
md.pageCount = utils.xlate(get('PageCount'), True)
md.scanInfo = utils.xlate(get('ScanInformation'))
md.storyArc = utils.xlate(get('StoryArc'))
md.seriesGroup = utils.xlate(get('SeriesGroup'))
md.maturityRating = utils.xlate(get('AgeRating'))
tmp = utils.xlate(get('BlackAndWhite'))
if tmp is not None and tmp.lower() in ["yes", "true", "1"]:
md.blackAndWhite = True
# Now extract the credit info
@ -261,23 +260,23 @@ class ComicInfoXml:
):
if n.text is not None:
for name in n.text.split(','):
metadata.addCredit(name.strip(), n.tag)
md.addCredit(name.strip(), n.tag)
if n.tag == 'CoverArtist':
if n.text is not None:
for name in n.text.split(','):
metadata.addCredit(name.strip(), "Cover")
md.addCredit(name.strip(), "Cover")
# parse page data now
pages_node = root.find("Pages")
if pages_node is not None:
for page in pages_node:
metadata.pages.append(page.attrib)
md.pages.append(page.attrib)
# print page.attrib
metadata.isEmpty = False
md.isEmpty = False
return metadata
return md
def writeToExternalFile(self, filename, metadata):

View File

@ -65,6 +65,7 @@ class GenericMetadata:
self.issue = None
self.title = None
self.publisher = None
self.seriesYear = None
self.month = None
self.year = None
self.day = None
@ -132,6 +133,7 @@ class GenericMetadata:
assign("issueCount", new_md.issueCount)
assign("title", new_md.title)
assign("publisher", new_md.publisher)
assign("seriesYear", new_md.seriesYear)
assign("day", new_md.day)
assign("month", new_md.month)
assign("year", new_md.year)
@ -263,6 +265,7 @@ class GenericMetadata:
add_attr_string("issueCount")
add_attr_string("title")
add_attr_string("publisher")
add_attr_string("seriesYear")
add_attr_string("year")
add_attr_string("month")
add_attr_string("day")

View File

@ -21,6 +21,7 @@ import re
import platform
import locale
import codecs
import unicodedata
class UtilsVars:
@ -121,6 +122,23 @@ def which(program):
return None
def xlate(data, isInt=False):
class Default(dict):
def __missing__(self, key):
return None
if data is None or data == "":
return None
if isInt:
i = str(data).translate(Default(zip((ord(c) for c in "1234567890"),"1234567890")))
if i == "0":
return "0"
if i is "":
return None
return int(i)
else:
return str(data)
def removearticles(text):
text = text.lower()
articles = ['and', 'a', '&', 'issue', 'the']
@ -131,19 +149,24 @@ def removearticles(text):
newText = newText[:-1]
# now get rid of some other junk
newText = newText.replace(":", "")
newText = newText.replace(",", "")
newText = newText.replace("-", " ")
# since the CV API changed, searches for series names with periods
# now explicitly require the period to be in the search key,
# so the line below is removed (for now)
#newText = newText.replace(".", "")
return newText
def sanitize_title(text):
# normalize unicode and convert to ascii. Does not work for everything eg ½ to 12 not 1/2
# this will probably cause issues with titles in other character sets e.g. chinese, japanese
text = unicodedata.normalize('NFKD', text).encode('ascii', 'ignore').decode('ascii')
# comicvine keeps apostrophes a part of the word
text = text.replace("'", "")
text = text.replace("\"", "")
# comicvine ignores punctuation and accents
text = re.sub(r'[^A-Za-z0-9]+',' ', text)
# remove extra space and articles and all lower case
text = removearticles(text).lower().strip()
return text
def unique_file(file_name):
counter = 1
# returns ('/path/file', '.ext')

View File

@ -9,13 +9,6 @@ binaries = []
block_cipher = None
if platform.system() == "Windows":
from site import getsitepackages
sitepackages = getsitepackages()[1]
# add ssl qt libraries not discovered automatically
binaries.extend([
(join(sitepackages, "PyQt5/Qt/bin/libeay32.dll"), "./PyQt5/Qt/bin"),
(join(sitepackages, "PyQt5/Qt/bin/ssleay32.dll"), "./PyQt5/Qt/bin")
])
enable_console = True
a = Analysis(['comictagger.py'],
@ -54,4 +47,4 @@ app = BUNDLE(exe,
'CFBundleShortVersionString': ctversion.version,
'CFBundleVersion': ctversion.version
},
bundle_identifier=None)
bundle_identifier=None)

View File

@ -507,19 +507,31 @@ def process_file_cli(filename, opts, settings, match_results):
renamer.setTemplate(settings.rename_template)
renamer.setIssueZeroPadding(settings.rename_issue_number_padding)
renamer.setSmartCleanup(settings.rename_use_smart_string_cleanup)
renamer.move = settings.rename_move_dir
new_name = renamer.determineName(filename, ext=new_ext)
if new_name == os.path.basename(filename):
print(msg_hdr + "Filename is already good!", file=sys.stderr)
try:
new_name = renamer.determineName(filename, ext=new_ext)
except Exception as e:
print(msg_hdr + "Invalid format string!\nYour rename template is invalid!\n\n"
"{}\n\nPlease consult the template help in the settings "
"and the documentation on the format at "
"https://docs.python.org/3/library/string.html#format-string-syntax", file=sys.stderr)
return
folder = os.path.dirname(os.path.abspath(filename))
if settings.rename_move_dir and len(settings.rename_dir.strip()) > 3:
folder = settings.rename_dir.strip()
new_abs_path = utils.unique_file(os.path.join(folder, new_name))
if os.path.join(folder, new_name) == os.path.abspath(filename):
print(msg_hdr + "Filename is already good!", file=sys.stderr)
return
suffix = ""
if not opts.dryrun:
# rename the file
os.makedirs(os.path.dirname(new_abs_path), 0o777, True)
os.rename(filename, new_abs_path)
else:
suffix = " (dry-run, no change)"

View File

@ -15,8 +15,7 @@
# limitations under the License.
import json
import urllib.request, urllib.error, urllib.parse
import urllib.request, urllib.parse, urllib.error
import requests
import re
import time
import datetime
@ -104,9 +103,6 @@ class ComicVineTalker(QObject):
self.log_func = None
# always use a tls context for urlopen
self.ssl = ssl.SSLContext(ssl.PROTOCOL_TLS)
def setLogFunc(self, log_func):
self.log_func = log_func
@ -124,23 +120,20 @@ class ComicVineTalker(QObject):
year = None
if date_str is not None:
parts = date_str.split('-')
year = parts[0]
year = utils.xlate(parts[0], True)
if len(parts) > 1:
month = parts[1]
month = utils.xlate(parts[1], True)
if len(parts) > 2:
day = parts[2]
day = utils.xlate(parts[2], True)
return day, month, year
def testKey(self, key):
try:
test_url = self.api_base_url + "/issue/1/?api_key=" + \
key + "&format=json&field_list=name"
resp = urllib.request.urlopen(test_url, context=self.ssl)
content = resp.read()
cv_response = json.loads(content.decode('utf-8'))
test_url = self.api_base_url + "/issue/1/?api_key=" + key + "&format=json&field_list=name"
cv_response = requests.get(test_url, headers={'user-agent': 'comictagger/' + ctversion.version}).json()
# Bogus request, but if the key is wrong, you get error 100: "Invalid
# API Key"
return cv_response['status_code'] != 100
@ -152,14 +145,13 @@ class ComicVineTalker(QObject):
sleep for a bit and retry.
"""
def getCVContent(self, url):
def getCVContent(self, url, params):
total_time_waited = 0
limit_wait_time = 1
counter = 0
wait_times = [1, 2, 3, 4]
while True:
content = self.getUrlContent(url)
cv_response = json.loads(content.decode('utf-8'))
cv_response = self.getUrlContent(url, params)
if self.wait_for_rate_limit and cv_response[
'status_code'] == ComicVineTalkerException.RateLimit:
self.writeLog(
@ -184,25 +176,24 @@ class ComicVineTalker(QObject):
break
return cv_response
def getUrlContent(self, url):
def getUrlContent(self, url, params):
# connect to server:
# if there is a 500 error, try a few more times before giving up
# any other error, just bail
#print("---", url)
for tries in range(3):
try:
resp = urllib.request.urlopen(url, context=self.ssl)
return resp.read()
except urllib.error.HTTPError as e:
if e.getcode() == 500:
resp = requests.get(url, params=params, headers={'user-agent': 'comictagger/' + ctversion.version})
if resp.status_code == 200:
return resp.json()
if resp.status_code == 500:
self.writeLog("Try #{0}: ".format(tries + 1))
time.sleep(1)
self.writeLog(str(e) + "\n")
if e.getcode() != 500:
self.writeLog(str(resp.status_code) + "\n")
else:
break
except Exception as e:
except requests.exceptions.RequestException as e:
self.writeLog(str(e) + "\n")
raise ComicVineTalkerException(
ComicVineTalkerException.Network, "Network Error!")
@ -212,8 +203,8 @@ class ComicVineTalker(QObject):
def searchForSeries(self, series_name, callback=None, refresh_cache=False):
# remove cruft from the search string
series_name = utils.removearticles(series_name).lower().strip()
# Sanitize the series name for comicvine searching, comicvine search ignore symbols
search_series_name = utils.sanitize_title(series_name)
# before we search online, look in our cache, since we might have
# done this same search recently
@ -224,19 +215,17 @@ class ComicVineTalker(QObject):
if len(cached_search_results) > 0:
return cached_search_results
original_series_name = series_name
params = {
'api_key': self.api_key,
'format': 'json',
'resources': 'volume',
'query': search_series_name,
'field_list': 'volume,name,id,start_year,publisher,image,description,count_of_issues',
'page': 1,
'limit': 100
}
# Split and rejoin to remove extra internal spaces
query_word_list = series_name.split()
query_string = " ".join( query_word_list ).strip()
#print ("Query string = ", query_string)
query_string = urllib.parse.quote_plus(query_string.encode("utf-8"))
search_url = self.api_base_url + "/search/?api_key=" + self.api_key + "&format=json&resources=volume&query=" + \
query_string + \
"&field_list=name,id,start_year,publisher,image,description,count_of_issues&limit=100"
cv_response = self.getCVContent(search_url + "&page=1")
cv_response = self.getCVContent(self.api_base_url + "/search", params)
search_results = list()
@ -249,15 +238,15 @@ class ComicVineTalker(QObject):
# 8 Dec 2018 - Comic Vine changed query results again. Terms are now
# ORed together, and we get thousands of results. Good news is the
# results are sorted by relevance, so we can be smart about halting
# the search.
# the search.
# 1. Don't fetch more than some sane amount of pages.
max_results = 500
max_results = 500
# 2. Halt when not all of our search terms are present in a result
# 3. Halt when the results contain more (plus threshold) words than
# our search
result_word_count_max = len(query_word_list) + 3
result_word_count_max = len(search_series_name.split()) + 3
total_result_count = min(total_result_count, max_results)
total_result_count = min(total_result_count, max_results)
if callback is None:
self.writeLog(
@ -276,18 +265,20 @@ class ComicVineTalker(QObject):
last_result = search_results[-1]['name']
# Sanitize the series name for comicvine searching, comicvine search ignore symbols
last_result = utils.sanitize_title(last_result)
# See if the last result's name has all the of the search terms.
# if not, break out of this, loop, we're done.
#print("Searching for {} in '{}'".format(query_word_list, last_result))
for term in query_word_list:
for term in search_series_name.split():
if term not in last_result.lower():
#print("Term '{}' not in last result. Halting search result fetching".format(term))
stop_searching = True
break
# Also, stop searching when the word count of last results is too much longer
# than our search terms list
if len(utils.removearticles(last_result).split()) > result_word_count_max:
# than our search terms list
if len(last_result) > result_word_count_max:
#print("Last result '{}' is too long. Halting search result fetching".format(last_result))
stop_searching = True
@ -301,7 +292,8 @@ class ComicVineTalker(QObject):
total_result_count))
page += 1
cv_response = self.getCVContent(search_url + "&page=" + str(page))
params['page'] = page
cv_response = self.getCVContent(self.api_base_url + "/search", params)
search_results.extend(cv_response['results'])
current_result_count += cv_response['number_of_page_results']
@ -313,8 +305,11 @@ class ComicVineTalker(QObject):
# (iterate backwards for easy removal)
for i in range(len(search_results) - 1, -1, -1):
record = search_results[i]
for term in query_word_list:
if term not in record['name'].lower():
# Sanitize the series name for comicvine searching, comicvine search ignore symbols
recordName = utils.sanitize_title(record['name'])
for term in search_series_name.split():
if term not in recordName:
del search_results[i]
break
@ -325,7 +320,7 @@ class ComicVineTalker(QObject):
#print(u"{0}: {1} ({2})".format(search_results['results'][0]['id'], search_results['results'][0]['name'] , search_results['results'][0]['start_year']))
# cache these search results
cvc.add_search_results(original_series_name, search_results)
cvc.add_search_results(series_name, search_results)
return search_results
@ -339,11 +334,14 @@ class ComicVineTalker(QObject):
if cached_volume_result is not None:
return cached_volume_result
volume_url = self.api_base_url + "/volume/" + CVTypeID.Volume + "-" + \
str(series_id) + "/?api_key=" + self.api_key + \
"&field_list=name,id,start_year,publisher,count_of_issues&format=json"
volume_url = self.api_base_url + "/volume/" + CVTypeID.Volume + "-" + str(series_id)
cv_response = self.getCVContent(volume_url)
params = {
'api_key': self.api_key,
'format': 'json',
'field_list': 'name,id,start_year,publisher,count_of_issues'
}
cv_response = self.getCVContent(volume_url, params)
volume_results = cv_response['results']
@ -361,11 +359,13 @@ class ComicVineTalker(QObject):
if cached_volume_issues_result is not None:
return cached_volume_issues_result
#---------------------------------
issues_url = self.api_base_url + "/issues/" + "?api_key=" + self.api_key + "&filter=volume:" + \
str(series_id) + \
"&field_list=id,volume,issue_number,name,image,cover_date,site_detail_url,description&format=json"
cv_response = self.getCVContent(issues_url)
params = {
'api_key': self.api_key,
'filter': 'volume:' + str(series_id),
'format': 'json',
'field_list': 'id,volume,issue_number,name,image,cover_date,site_detail_url,description'
}
cv_response = self.getCVContent(self.api_base_url + "/issues/", params)
#------------------------------------
@ -385,9 +385,8 @@ class ComicVineTalker(QObject):
page += 1
offset += cv_response['number_of_page_results']
# print issues_url+ "&offset="+str(offset)
cv_response = self.getCVContent(
issues_url + "&offset=" + str(offset))
params['offset'] = offset
cv_response = self.getCVContent(self.api_base_url + "/issues/", params)
volume_issues_result.extend(cv_response['results'])
current_result_count += cv_response['number_of_page_results']
@ -398,26 +397,24 @@ class ComicVineTalker(QObject):
return volume_issues_result
def fetchIssuesByVolumeIssueNumAndYear(
self, volume_id_list, issue_number, year):
volume_filter = "volume:"
def fetchIssuesByVolumeIssueNumAndYear(self, volume_id_list, issue_number, year):
volume_filter = ""
for vid in volume_id_list:
volume_filter += str(vid) + "|"
filter = "volume:{},issue_number:{}".format(volume_filter, issue_number)
year_filter = ""
if year is not None and str(year).isdigit():
year_filter = ",cover_date:{0}-1-1|{1}-1-1".format(
year, int(year) + 1)
intYear = utils.xlate(year, True)
if intYear is not None:
filter += ",cover_date:{}-1-1|{}-1-1".format(intYear, intYear + 1)
issue_number = urllib.parse.quote_plus(str(issue_number).encode("utf-8"))
params = {
'api_key': self.api_key,
'format': 'json',
'field_list': 'id,volume,issue_number,name,image,cover_date,site_detail_url,description',
'filter': filter
}
filter = "&filter=" + volume_filter + \
year_filter + ",issue_number:" + issue_number
issues_url = self.api_base_url + "/issues/" + "?api_key=" + self.api_key + filter + \
"&field_list=id,volume,issue_number,name,image,cover_date,site_detail_url,description&format=json"
cv_response = self.getCVContent(issues_url)
cv_response = self.getCVContent(self.api_base_url + "/issues", params)
#------------------------------------
@ -437,9 +434,8 @@ class ComicVineTalker(QObject):
page += 1
offset += cv_response['number_of_page_results']
# print issues_url+ "&offset="+str(offset)
cv_response = self.getCVContent(
issues_url + "&offset=" + str(offset))
params['offset'] = offset
cv_response = self.getCVContent(self.api_base_url + "/issues/", params)
filtered_issues_result.extend(cv_response['results'])
current_result_count += cv_response['number_of_page_results']
@ -463,11 +459,12 @@ class ComicVineTalker(QObject):
break
if (found):
issue_url = self.api_base_url + "/issue/" + CVTypeID.Issue + "-" + \
str(record['id']) + "/?api_key=" + \
self.api_key + "&format=json"
cv_response = self.getCVContent(issue_url)
issue_url = self.api_base_url + "/issue/" + CVTypeID.Issue + "-" + str(record['id'])
params = {
'api_key': self.api_key,
'format': 'json'
}
cv_response = self.getCVContent(issue_url, params)
issue_results = cv_response['results']
else:
@ -479,9 +476,12 @@ class ComicVineTalker(QObject):
def fetchIssueDataByIssueID(self, issue_id, settings):
issue_url = self.api_base_url + "/issue/" + CVTypeID.Issue + "-" + \
str(issue_id) + "/?api_key=" + self.api_key + "&format=json"
cv_response = self.getCVContent(issue_url)
issue_url = self.api_base_url + "/issue/" + CVTypeID.Issue + "-" + str(issue_id)
params = {
'api_key': self.api_key,
'format': 'json'
}
cv_response = self.getCVContent(issue_url, params)
issue_results = cv_response['results']
@ -497,21 +497,20 @@ class ComicVineTalker(QObject):
# Now, map the Comic Vine data to generic metadata
metadata = GenericMetadata()
metadata.series = issue_results['volume']['name']
metadata.series = utils.xlate(issue_results['volume']['name'])
metadata.issue = IssueString(issue_results['issue_number']).asString()
metadata.title = utils.xlate(issue_results['name'])
num_s = IssueString(issue_results['issue_number']).asString()
metadata.issue = num_s
metadata.title = issue_results['name']
if volume_results['publisher'] is not None:
metadata.publisher = utils.xlate(volume_results['publisher']['name'])
metadata.day, metadata.month, metadata.year = self.parseDateStr(issue_results['cover_date'])
metadata.publisher = volume_results['publisher']['name']
metadata.day, metadata.month, metadata.year = self.parseDateStr(
issue_results['cover_date'])
#metadata.issueCount = volume_results['count_of_issues']
metadata.seriesYear = utils.xlate(volume_results['start_year'])
metadata.issueCount = utils.xlate(volume_results['count_of_issues'])
metadata.comments = self.cleanup_html(
issue_results['description'], settings.remove_html_tables)
if settings.use_series_start_as_volume:
metadata.volume = volume_results['start_year']
metadata.volume = utils.xlate(volume_results['start_year'])
metadata.notes = "Tagged with ComicTagger {0} using info from Comic Vine on {1}. [Issue ID {2}]".format(
ctversion.version,
@ -672,9 +671,15 @@ class ComicVineTalker(QObject):
if cached_details['image_url'] is not None:
return cached_details
issue_url = self.api_base_url + "/issue/" + CVTypeID.Issue + "-" + \
str(issue_id) + "/?api_key=" + self.api_key + \
"&format=json&field_list=image,cover_date,site_detail_url"
issue_url = self.api_base_url + "/issue/" + CVTypeID.Issue + "-" + str(issue_id)
params = {
'api_key': self.api_key,
'format': 'json',
'field_list': 'image,cover_date,site_detail_url'
}
cv_response = self.getCVContent(issue_url, params)
details = dict()
details['image_url'] = None
@ -682,8 +687,6 @@ class ComicVineTalker(QObject):
details['cover_date'] = None
details['site_detail_url'] = None
cv_response = self.getCVContent(issue_url)
details['image_url'] = cv_response['results']['image']['super_url']
details['thumb_image_url'] = cv_response[
'results']['image']['thumb_url']
@ -718,8 +721,7 @@ class ComicVineTalker(QObject):
return url_list
# scrape the CV issue page URL to get the alternate cover URLs
resp = urllib.request.urlopen(issue_page_url, context=self.ssl)
content = resp.read()
content = requests.get(issue_page_url, headers={'user-agent': 'comictagger/' + ctversion.version}).text
alt_cover_url_list = self.parseOutAltCoverUrls(content)
# cache this alt cover URL list
@ -729,9 +731,9 @@ class ComicVineTalker(QObject):
def parseOutAltCoverUrls(self, page_html):
soup = BeautifulSoup(page_html, "html.parser")
alt_cover_url_list = []
# Using knowledge of the layout of the Comic Vine issue page here:
# look for the divs that are in the classes 'imgboxart' and
# 'issue-cover'
@ -740,15 +742,15 @@ class ComicVineTalker(QObject):
for d in div_list:
if 'class' in d.attrs:
c = d['class']
if ('imgboxart' in c and
if ('imgboxart' in c and
'issue-cover' in c and
d.img['src'].startswith("http")
):
covers_found += 1
if covers_found != 1:
alt_cover_url_list.append(d.img['src'])
return alt_cover_url_list
def fetchCachedAlternateCoverURLs(self, issue_id):

View File

@ -17,19 +17,96 @@
import os
import re
import datetime
import sys
import string
from pathvalidate import sanitize_filepath
from . import utils
from .issuestring import IssueString
class MetadataFormatter(string.Formatter):
def __init__(self, smart_cleanup=False):
super().__init__()
self.smart_cleanup = smart_cleanup
def format_field(self, value, format_spec):
if value is None or value == "":
return ""
return super().format_field(value, format_spec)
def _vformat(self, format_string, args, kwargs, used_args, recursion_depth,
auto_arg_index=0):
if recursion_depth < 0:
raise ValueError('Max string recursion exceeded')
result = []
lstrip = False
for literal_text, field_name, format_spec, conversion in \
self.parse(format_string):
# output the literal text
if literal_text:
if lstrip:
result.append(literal_text.lstrip("-_)}]#"))
else:
result.append(literal_text)
lstrip = False
# if there's a field, output it
if field_name is not None:
# this is some markup, find the object and do
# the formatting
# handle arg indexing when empty field_names are given.
if field_name == '':
if auto_arg_index is False:
raise ValueError('cannot switch from manual field '
'specification to automatic field '
'numbering')
field_name = str(auto_arg_index)
auto_arg_index += 1
elif field_name.isdigit():
if auto_arg_index:
raise ValueError('cannot switch from manual field '
'specification to automatic field '
'numbering')
# disable auto arg incrementing, if it gets
# used later on, then an exception will be raised
auto_arg_index = False
# given the field_name, find the object it references
# and the argument it came from
obj, arg_used = self.get_field(field_name, args, kwargs)
used_args.add(arg_used)
# do any conversion on the resulting object
obj = self.convert_field(obj, conversion)
# expand the format spec, if needed
format_spec, auto_arg_index = self._vformat(
format_spec, args, kwargs,
used_args, recursion_depth-1,
auto_arg_index=auto_arg_index)
# format the object and append to the result
fmtObj = self.format_field(obj, format_spec)
if fmtObj == "" and len(result) > 0 and self.smart_cleanup:
lstrip = True
result.pop()
result.append(fmtObj)
return ''.join(result), auto_arg_index
class FileRenamer:
def __init__(self, metadata):
self.setMetadata(metadata)
self.setTemplate(
"%series% v%volume% #%issue% (of %issuecount%) (%year%)")
"{publisher}/{series}/{series} v{volume} #{issue} (of {issueCount}) ({year})")
self.smart_cleanup = True
self.issue_zero_padding = 3
self.move = False
def setMetadata(self, metadata):
self.metdata = metadata
@ -43,114 +120,37 @@ class FileRenamer:
def setTemplate(self, template):
self.template = template
def replaceToken(self, text, value, token):
# helper func
def isToken(word):
return (word[0] == "%" and word[-1:] == "%")
if value is not None:
return text.replace(token, str(value))
else:
if self.smart_cleanup:
# smart cleanup means we want to remove anything appended to token if it's empty
# (e.g "#%issue%" or "v%volume%")
# (TODO: This could fail if there is more than one token appended together, I guess)
text_list = text.split()
# special case for issuecount, remove preceding non-token word,
# as in "...(of %issuecount%)..."
if token == '%issuecount%':
for idx, word in enumerate(text_list):
if token in word and not isToken(text_list[idx - 1]):
text_list[idx - 1] = ""
text_list = [x for x in text_list if token not in x]
return " ".join(text_list)
else:
return text.replace(token, "")
def determineName(self, filename, ext=None):
class Default(dict):
def __missing__(self, key):
return "{" + key + "}"
md = self.metdata
new_name = self.template
preferred_encoding = utils.get_actual_preferred_encoding()
# print(u"{0}".format(md))
new_name = self.replaceToken(new_name, md.series, '%series%')
new_name = self.replaceToken(new_name, md.volume, '%volume%')
# padding for issue
md.issue = IssueString(md.issue).asString(pad=self.issue_zero_padding)
if md.issue is not None:
issue_str = "{0}".format(
IssueString(md.issue).asString(pad=self.issue_zero_padding))
else:
issue_str = None
new_name = self.replaceToken(new_name, issue_str, '%issue%')
template = self.template
new_name = self.replaceToken(new_name, md.issueCount, '%issuecount%')
new_name = self.replaceToken(new_name, md.year, '%year%')
new_name = self.replaceToken(new_name, md.publisher, '%publisher%')
new_name = self.replaceToken(new_name, md.title, '%title%')
new_name = self.replaceToken(new_name, md.month, '%month%')
month_name = None
if md.month is not None:
if (isinstance(md.month, str) and md.month.isdigit()) or isinstance(
md.month, int):
if int(md.month) in range(1, 13):
dt = datetime.datetime(1970, int(md.month), 1, 0, 0)
#month_name = dt.strftime("%B".encode(preferred_encoding)).decode(preferred_encoding)
month_name = dt.strftime("%B")
new_name = self.replaceToken(new_name, month_name, '%month_name%')
pathComponents = template.split(os.sep)
new_name = ""
new_name = self.replaceToken(new_name, md.genre, '%genre%')
new_name = self.replaceToken(new_name, md.language, '%language_code%')
new_name = self.replaceToken(
new_name, md.criticalRating, '%criticalrating%')
new_name = self.replaceToken(
new_name, md.alternateSeries, '%alternateseries%')
new_name = self.replaceToken(
new_name, md.alternateNumber, '%alternatenumber%')
new_name = self.replaceToken(
new_name, md.alternateCount, '%alternatecount%')
new_name = self.replaceToken(new_name, md.imprint, '%imprint%')
new_name = self.replaceToken(new_name, md.format, '%format%')
new_name = self.replaceToken(
new_name, md.maturityRating, '%maturityrating%')
new_name = self.replaceToken(new_name, md.storyArc, '%storyarc%')
new_name = self.replaceToken(new_name, md.seriesGroup, '%seriesgroup%')
new_name = self.replaceToken(new_name, md.scanInfo, '%scaninfo%')
fmt = MetadataFormatter(self.smart_cleanup)
for Component in pathComponents:
new_name = os.path.join(new_name, fmt.vformat(Component, args=[], kwargs=Default(vars(md))).replace("/", "-"))
if self.smart_cleanup:
# remove empty braces,brackets, parentheses
new_name = re.sub("\(\s*[-:]*\s*\)", "", new_name)
new_name = re.sub("\[\s*[-:]*\s*\]", "", new_name)
new_name = re.sub("\{\s*[-:]*\s*\}", "", new_name)
# remove duplicate spaces
new_name = " ".join(new_name.split())
# remove remove duplicate -, _,
new_name = re.sub("[-_]{2,}\s+", "-- ", new_name)
new_name = re.sub("(\s--)+", " --", new_name)
new_name = re.sub("(\s-)+", " -", new_name)
# remove dash or double dash at end of line
new_name = re.sub("[-]{1,2}\s*$", "", new_name)
# remove duplicate spaces (again!)
new_name = " ".join(new_name.split())
if ext is None:
if ext is None or ext == "":
ext = os.path.splitext(filename)[1]
new_name += ext
# some tweaks to keep various filesystems happy
new_name = new_name.replace("/", "-")
new_name = new_name.replace(" :", " -")
new_name = new_name.replace(": ", " - ")
new_name = new_name.replace(":", "-")
new_name = new_name.replace("?", "")
return new_name
# remove padding
md.issue = IssueString(md.issue).asString()
if self.move:
return sanitize_filepath(new_name.strip())
else:
return os.path.basename(sanitize_filepath(new_name.strip()))

View File

@ -19,9 +19,7 @@ import os
import datetime
import shutil
import tempfile
import urllib.request, urllib.parse, urllib.error
import ssl
#import urllib2
import requests
try:
from PyQt5.QtNetwork import QNetworkAccessManager, QNetworkRequest
@ -46,6 +44,7 @@ except ImportError:
pass
from .settings import ComicTaggerSettings
from . import ctversion
class ImageFetcherException(Exception):
@ -66,9 +65,6 @@ class ImageFetcher(QObject):
if not os.path.exists(self.db_file):
self.create_image_db()
# always use a tls context for urlopen
self.ssl = ssl.SSLContext(ssl.PROTOCOL_TLS)
def clearCache(self):
os.unlink(self.db_file)
if os.path.isdir(self.cache_folder):
@ -90,7 +86,8 @@ class ImageFetcher(QObject):
if blocking:
if image_data is None:
try:
image_data = urllib.request.urlopen(url, context=self.ssl).read()
print(url)
image_data = requests.get(url, headers={'user-agent': 'comictagger/' + ctversion.version}).content
except Exception as e:
print(e)
raise ImageFetcherException("Network Error!")

View File

@ -51,7 +51,6 @@ class ImageHasher(object):
image = self.image.resize(
(self.width, self.height), Image.ANTIALIAS).convert("L")
except Exception as e:
sys.exc_clear()
print("average_hash error:", e)
return int(0)

View File

@ -16,9 +16,6 @@
import sys
import io
#import math
#import urllib2
#import urllib
try:
from PIL import Image
@ -138,7 +135,6 @@ class IssueIdentifier:
try:
cropped_im = im.crop((int(w / 2), 0, w, h))
except Exception as e:
sys.exc_clear()
print("cropCover() error:", e)
return None
@ -438,8 +434,10 @@ class IssueIdentifier:
# assume that our search name is close to the actual name, say
# within ,e.g. 5 chars
shortened_key = utils.removearticles(keys['series'])
shortened_item_name = utils.removearticles(item['name'])
# sanitize both the search string and the result so that
# we are comparing the same type of data
shortened_key = utils.sanitize_title(keys['series'])
shortened_item_name = utils.sanitize_title(item['name'])
if len(shortened_item_name) < (
len(shortened_key) + self.length_delta_thresh):
length_approved = True

View File

@ -157,7 +157,7 @@ class PageListEditor(QWidget):
self.modified.emit()
def changePageType(self, i):
new_type = self.comboBox.itemData(i).toString()
new_type = self.comboBox.itemData(i)
if self.getCurrentPageType() != new_type:
self.setCurrentPageType(new_type)
self.emitFrontCoverChange()

View File

@ -76,7 +76,20 @@ class RenameWindow(QtWidgets.QDialog):
if md.isEmpty:
md = ca.metadataFromFilename(self.settings.parse_scan_info)
self.renamer.setMetadata(md)
new_name = self.renamer.determineName(ca.path, ext=new_ext)
self.renamer.move = self.settings.rename_move_dir
try:
new_name = self.renamer.determineName(ca.path, ext=new_ext)
except Exception as e:
QtWidgets.QMessageBox.critical(self, 'Invalid format string!',
'Your rename template is invalid!'
'<br/><br/>{}<br/><br/>'
'Please consult the template help in the '
'settings and the documentation on the format at '
'<a href=\'https://docs.python.org/3/library/string.html#format-string-syntax\'>'
'https://docs.python.org/3/library/string.html#format-string-syntax</a>'.format(e))
return
row = self.twList.rowCount()
self.twList.insertRow(row)
@ -149,17 +162,20 @@ class RenameWindow(QtWidgets.QDialog):
centerWindowOnParent(progdialog)
QtCore.QCoreApplication.processEvents()
if item['new_name'] == os.path.basename(item['archive'].path):
folder = os.path.dirname(os.path.abspath(item['archive'].path))
if self.settings.rename_move_dir and len(self.settings.rename_dir.strip()) > 3:
folder = self.settings.rename_dir.strip()
new_abs_path = utils.unique_file(os.path.join(folder, item['new_name']))
if os.path.join(folder, item['new_name']) == item['archive'].path:
print(item['new_name'], "Filename is already good!")
continue
if not item['archive'].isWritable(check_rar_status=False):
continue
folder = os.path.dirname(os.path.abspath(item['archive'].path))
new_abs_path = utils.unique_file(
os.path.join(folder, item['new_name']))
os.makedirs(os.path.dirname(new_abs_path), 0o777, True)
os.rename(item['archive'].path, new_abs_path)
item['archive'].rename(new_abs_path)

View File

@ -108,10 +108,12 @@ class ComicTaggerSettings:
self.apply_cbl_transform_on_bulk_operation = False
# Rename settings
self.rename_template = "%series% #%issue% (%year%)"
self.rename_template = "{publisher}/{series}/{series} #{issue} - {title} ({year})"
self.rename_issue_number_padding = 3
self.rename_use_smart_string_cleanup = True
self.rename_extension_based_on_archive = True
self.rename_dir = ""
self.rename_move_dir = False
# Auto-tag stickies
self.save_on_low_confidence = False
@ -301,6 +303,10 @@ class ComicTaggerSettings:
'rename', 'rename_extension_based_on_archive'):
self.rename_extension_based_on_archive = self.config.getboolean(
'rename', 'rename_extension_based_on_archive')
if self.config.has_option('rename', 'rename_dir'):
self.rename_dir = self.config.get('rename', 'rename_dir')
if self.config.has_option('rename', 'rename_move_dir'):
self.rename_move_dir = self.config.getboolean('rename', 'rename_move_dir')
if self.config.has_option('autotag', 'save_on_low_confidence'):
self.save_on_low_confidence = self.config.getboolean(
@ -462,6 +468,8 @@ class ComicTaggerSettings:
self.rename_use_smart_string_cleanup)
self.config.set('rename', 'rename_extension_based_on_archive',
self.rename_extension_based_on_archive)
self.config.set('rename', 'rename_dir', self.rename_dir)
self.config.set('rename', 'rename_move_dir', self.rename_move_dir)
if not self.config.has_section('autotag'):
self.config.add_section('autotag')

View File

@ -24,6 +24,8 @@ from .settings import ComicTaggerSettings
from .comicvinecacher import ComicVineCacher
from .comicvinetalker import ComicVineTalker
from .imagefetcher import ImageFetcher
from .filerenamer import FileRenamer
from .genericmetadata import GenericMetadata
from . import utils
@ -113,6 +115,66 @@ class SettingsWindow(QtWidgets.QDialog):
self.btnClearCache.clicked.connect(self.clearCache)
self.btnResetSettings.clicked.connect(self.resetSettings)
self.btnTestKey.clicked.connect(self.testAPIKey)
self.btnTemplateHelp.clicked.connect(self.showTemplateHelp)
def configRenamer(self):
md = GenericMetadata()
md.isEmpty = False
md.tagOrigin = "testing"
md.series = "series name"
md.issue = "1"
md.title = "issue title"
md.publisher = "publisher"
md.year = 1998
md.month = 4
md.day = 4
md.issueCount = 1
md.volume = 256
md.genre = "test"
md.language = "en" # 2 letter iso code
md.comments = "This is definitly a comic." # use same way as Summary in CIX
md.volumeCount = 4096
md.criticalRating = "Worst Comic Ever"
md.country = "US"
md.alternateSeries = "None"
md.alternateNumber = 4.4
md.alternateCount = 4444
md.imprint = 'imprint'
md.notes = "This doesn't actually exist"
md.webLink = "https://example.com/series name/1"
md.format = "Box Set"
md.manga = "Yes"
md.blackAndWhite = False
md.pageCount = 4
md.maturityRating = "Everyone"
md.storyArc = "story"
md.seriesGroup = "seriesGroup"
md.scanInfo = "(lordwelch)"
md.characters = "character 1, character 2"
md.teams = "None"
md.locations = "Earth, 444 B.C."
md.credits = [dict({'role': 'Everything', 'person': 'author', 'primary': True})]
md.tags = ["testing", "not real"]
md.pages = [dict({'Image': '0', 'Type': 'Front Cover'}), dict({'Image': '1', 'Type': 'Story'})]
# Some CoMet-only items
md.price = 0.00
md.isVersionOf = "SERIES #1"
md.rights = "None"
md.identifier = "LW4444-Comic"
md.lastMark = "0"
md.coverImage = "https://example.com/series name/1/cover"
self.renamer = FileRenamer(md)
self.renamer.setTemplate(str(self.leRenameTemplate.text()))
self.renamer.setIssueZeroPadding(self.settings.rename_issue_number_padding)
self.renamer.setSmartCleanup(self.settings.rename_use_smart_string_cleanup)
def settingsToForm(self):
@ -165,9 +227,27 @@ class SettingsWindow(QtWidgets.QDialog):
self.cbxSmartCleanup.setCheckState(QtCore.Qt.Checked)
if self.settings.rename_extension_based_on_archive:
self.cbxChangeExtension.setCheckState(QtCore.Qt.Checked)
if self.settings.rename_move_dir:
self.cbxMoveFiles.setCheckState(QtCore.Qt.Checked)
self.leDirectory.setText(self.settings.rename_dir)
def accept(self):
self.configRenamer()
try:
new_name = self.renamer.determineName('test.cbz')
except Exception as e:
QtWidgets.QMessageBox.critical(self, 'Invalid format string!',
'Your rename template is invalid!'
'<br/><br/>{}<br/><br/>'
'Please consult the template help in the '
'settings and the documentation on the format at '
'<a href=\'https://docs.python.org/3/library/string.html#format-string-syntax\'>'
'https://docs.python.org/3/library/string.html#format-string-syntax</a>'.format(e))
return
# Copy values from form to settings and save
self.settings.rar_exe_path = str(self.leRarExePath.text())
@ -210,6 +290,8 @@ class SettingsWindow(QtWidgets.QDialog):
self.leIssueNumPadding.text())
self.settings.rename_use_smart_string_cleanup = self.cbxSmartCleanup.isChecked()
self.settings.rename_extension_based_on_archive = self.cbxChangeExtension.isChecked()
self.settings.rename_move_dir = self.cbxMoveFiles.isChecked()
self.settings.rename_dir = self.leDirectory.text()
self.settings.save()
QtWidgets.QDialog.accept(self)
@ -268,3 +350,17 @@ class SettingsWindow(QtWidgets.QDialog):
def showRenameTab(self):
self.tabWidget.setCurrentIndex(5)
def showTemplateHelp(self):
TemplateHelpWin = TemplateHelpWindow(self)
TemplateHelpWin.setModal(False)
TemplateHelpWin.show()
class TemplateHelpWindow(QtWidgets.QDialog):
def __init__(self, parent):
super(TemplateHelpWindow, self).__init__(parent)
uic.loadUi(ComicTaggerSettings.getUIFile('TemplateHelp.ui'), self)

View File

@ -51,6 +51,7 @@ from .cbltransformer import CBLTransformer
from .renamewindow import RenameWindow
from .exportwindow import ExportWindow, ExportConflictOpts
from .issueidentifier import IssueIdentifier
from .issuestring import IssueString
from .autotagstartwindow import AutoTagStartWindow
from .autotagprogresswindow import AutoTagProgressWindow
from .autotagmatchwindow import AutoTagMatchWindow
@ -186,6 +187,8 @@ class TaggerWindow(QtWidgets.QMainWindow):
validator = QtGui.QIntValidator(1900, 2099, self)
self.lePubYear.setValidator(validator)
self.leSeriesPubYear.setValidator(validator)
validator = QtGui.QIntValidator(1, 12, self)
self.lePubMonth.setValidator(validator)
@ -761,14 +764,12 @@ class TaggerWindow(QtWidgets.QMainWindow):
for child in widget.children():
self.clearChildren(child)
# Copy all of the metadata object into to the form.
# Merging of metadata should be done via the overlay function
def metadataToForm(self):
# copy the the metadata object into to the form
# helper func
def assignText(field, value):
if value is not None:
field.setText(str(value))
md = self.metadata
assignText(self.leSeries, md.series)
@ -781,6 +782,7 @@ class TaggerWindow(QtWidgets.QMainWindow):
assignText(self.lePubMonth, md.month)
assignText(self.lePubYear, md.year)
assignText(self.lePubDay, md.day)
assignText(self.leSeriesPubYear, md.seriesYear)
assignText(self.leGenre, md.genre)
assignText(self.leImprint, md.imprint)
assignText(self.teComments, md.comments)
@ -810,23 +812,33 @@ class TaggerWindow(QtWidgets.QMainWindow):
self.cbMaturityRating.setEditText(md.maturityRating)
else:
self.cbMaturityRating.setCurrentIndex(i)
else:
self.cbMaturityRating.setCurrentIndex(0)
if md.language is not None:
i = self.cbLanguage.findData(md.language)
self.cbLanguage.setCurrentIndex(i)
else:
self.cbLanguage.setCurrentIndex(0)
if md.country is not None:
i = self.cbCountry.findText(md.country)
self.cbCountry.setCurrentIndex(i)
else:
self.cbCountry.setCurrentIndex(0)
if md.manga is not None:
i = self.cbManga.findData(md.manga)
self.cbManga.setCurrentIndex(i)
else:
self.cbManga.setCurrentIndex(0)
if md.blackAndWhite is not None and md.blackAndWhite:
if md.blackAndWhite:
self.cbBW.setChecked(True)
else:
self.cbBW.setChecked(False)
assignText(self.teTags, utils.listToString(md.tags))
self.teTags.setText(utils.listToString(md.tags))
# !!! Should we clear the credits table or just avoid duplicates?
while self.twCredits.rowCount() > 0:
@ -885,58 +897,48 @@ class TaggerWindow(QtWidgets.QMainWindow):
return False
def formToMetadata(self):
# helper func
def xlate(data, type_str):
s = "{0}".format(data).strip()
if s == "":
return None
elif type_str == "str":
return s
else:
return int(s)
# copy the data from the form into the metadata
md = self.metadata
md.series = xlate(self.leSeries.text(), "str")
md.issue = xlate(self.leIssueNum.text(), "str")
md.issueCount = xlate(self.leIssueCount.text(), "int")
md.volume = xlate(self.leVolumeNum.text(), "int")
md.volumeCount = xlate(self.leVolumeCount.text(), "int")
md.title = xlate(self.leTitle.text(), "str")
md.publisher = xlate(self.lePublisher.text(), "str")
md.month = xlate(self.lePubMonth.text(), "int")
md.year = xlate(self.lePubYear.text(), "int")
md.day = xlate(self.lePubDay.text(), "int")
md.genre = xlate(self.leGenre.text(), "str")
md.imprint = xlate(self.leImprint.text(), "str")
md.comments = xlate(self.teComments.toPlainText(), "str")
md.notes = xlate(self.teNotes.toPlainText(), "str")
md.criticalRating = xlate(self.leCriticalRating.text(), "int")
md.maturityRating = xlate(self.cbMaturityRating.currentText(), "str")
md = GenericMetadata()
md.isEmpty = False
md.alternateNumber = IssueString(self.leAltIssueNum.text()).asString()
md.issue = IssueString(self.leIssueNum.text()).asString()
md.issueCount = utils.xlate(self.leIssueCount.text(), True)
md.volume = utils.xlate(self.leVolumeNum.text(), True)
md.volumeCount = utils.xlate(self.leVolumeCount.text(), True)
md.month = utils.xlate(self.lePubMonth.text(), True)
md.year = utils.xlate(self.lePubYear.text(), True)
md.day = utils.xlate(self.lePubDay.text(), True)
md.seriesYear = utils.xlate(self.leSeriesPubYear.text(), "int")
md.criticalRating = utils.xlate(self.leCriticalRating.text(), True)
md.alternateCount = utils.xlate(self.leAltIssueCount.text(), True)
md.storyArc = xlate(self.leStoryArc.text(), "str")
md.scanInfo = xlate(self.leScanInfo.text(), "str")
md.seriesGroup = xlate(self.leSeriesGroup.text(), "str")
md.alternateSeries = xlate(self.leAltSeries.text(), "str")
md.alternateNumber = xlate(self.leAltIssueNum.text(), "int")
md.alternateCount = xlate(self.leAltIssueCount.text(), "int")
md.webLink = xlate(self.leWebLink.text(), "str")
md.characters = xlate(self.teCharacters.toPlainText(), "str")
md.teams = xlate(self.teTeams.toPlainText(), "str")
md.locations = xlate(self.teLocations.toPlainText(), "str")
md.series = self.leSeries.text()
md.title = self.leTitle.text()
md.publisher = self.lePublisher.text()
md.genre = self.leGenre.text()
md.imprint = self.leImprint.text()
md.comments = self.teComments.toPlainText()
md.notes = self.teNotes.toPlainText()
md.maturityRating = self.cbMaturityRating.currentText()
md.format = xlate(self.cbFormat.currentText(), "str")
md.country = xlate(self.cbCountry.currentText(), "str")
md.storyArc = self.leStoryArc.text()
md.scanInfo = self.leScanInfo.text()
md.seriesGroup = self.leSeriesGroup.text()
md.alternateSeries = self.leAltSeries.text()
md.webLink = self.leWebLink.text()
md.characters = self.teCharacters.toPlainText()
md.teams = self.teTeams.toPlainText()
md.locations = self.teLocations.toPlainText()
langiso = self.cbLanguage.itemData(self.cbLanguage.currentIndex())
md.language = xlate(langiso, "str")
md.format = self.cbFormat.currentText()
md.country = self.cbCountry.currentText()
manga_code = self.cbManga.itemData(self.cbManga.currentIndex())
md.manga = xlate(manga_code, "str")
md.language = utils.xlate(self.cbLanguage.itemData(self.cbLanguage.currentIndex()))
md.manga = utils.xlate(self.cbManga.itemData(self.cbManga.currentIndex()))
# Make a list from the coma delimited tags string
tmp = xlate(self.teTags.toPlainText(), "str")
tmp = self.teTags.toPlainText()
if tmp is not None:
def striplist(l):
return([x.strip() for x in l])
@ -961,6 +963,8 @@ class TaggerWindow(QtWidgets.QMainWindow):
md.pages = self.pageListEditor.getPageList()
self.metadata = md
def useFilename(self):
if self.comic_archive is not None:
# copy the form onto metadata object

View File

@ -0,0 +1,107 @@
<?xml version="1.0" encoding="UTF-8"?>
<ui version="4.0">
<class>Dialog</class>
<widget class="QDialog" name="Dialog">
<property name="geometry">
<rect>
<x>0</x>
<y>0</y>
<width>702</width>
<height>452</height>
</rect>
</property>
<property name="windowTitle">
<string>Template Help</string>
</property>
<property name="sizeGripEnabled">
<bool>true</bool>
</property>
<layout class="QVBoxLayout" name="verticalLayout">
<property name="spacing">
<number>0</number>
</property>
<property name="leftMargin">
<number>2</number>
</property>
<property name="topMargin">
<number>2</number>
</property>
<item>
<widget class="QTextBrowser" name="textBrowser">
<property name="html">
<string>&lt;html&gt;
&lt;head/&gt;
&lt;body&gt;
&lt;h1 style=&quot;text-align: center&quot;&gt;Template help&lt;/h1&gt;
&lt;p&gt;The template uses Python format strings, in the simplest use it replaces the field (e.g. {issue}) with the value for that particular comic (e.g. 1) for advanced formatting please reference the
&lt;a href=&quot;https://docs.python.org/3/library/string.html#format-string-syntax&quot;&gt;Python 3 documentation&lt;/a&gt;&lt;/p&gt;
&lt;pre&gt;Accepts the following variables:
{isEmpty}&#009;&#009;(boolean)
{tagOrigin}&#009;&#009;(string)
{series}&#009;&#009;(string)
{issue}&#009;&#009;(string)
{title}&#009;&#009;(string)
{publisher}&#009;&#009;(string)
{month}&#009;&#009;(integer)
{year}&#009;&#009;(integer)
{day}&#009;&#009;(integer)
{issueCount}&#009;(integer)
{volume}&#009;&#009;(integer)
{genre}&#009;&#009;(string)
{language}&#009;&#009;(string)
{comments}&#009;&#009;(string)
{volumeCount}&#009;(integer)
{criticalRating}&#009;(string)
{country}&#009;&#009;(string)
{alternateSeries}&#009;(string)
{alternateNumber}&#009;(string)
{alternateCount}&#009;(integer)
{imprint}&#009;&#009;(string)
{notes}&#009;&#009;(string)
{webLink}&#009;&#009;(string)
{format}&#009;&#009;(string)
{manga}&#009;&#009;(string)
{blackAndWhite}&#009;(boolean)
{pageCount}&#009;&#009;(integer)
{maturityRating}&#009;(string)
{storyArc}&#009;&#009;(string)
{seriesGroup}&#009;(string)
{scanInfo}&#009;&#009;(string)
{characters}&#009;(string)
{teams}&#009;&#009;(string)
{locations}&#009;&#009;(string)
{credits}&#009;&#009;(list of dict({&apos;role&apos;: &apos;str&apos;, &apos;person&apos;: &apos;str&apos;, &apos;primary&apos;: boolean}))
{tags}&#009;&#009;(list of str)
{pages}&#009;&#009;(list of dict({&apos;Image&apos;: &apos;str(int)&apos;, &apos;Type&apos;: &apos;str&apos;}))
CoMet-only items:
{price}&#009;&#009;(float)
{isVersionOf}&#009;(string)
{rights}&#009;&#009;(string)
{identifier}&#009;(string)
{lastMark}&#009;(string)
{coverImage}&#009;(string)
Examples:
{series} {issue} ({year})
Spider-Geddon 1 (2018)
{series} #{issue} - {title}
Spider-Geddon #1 - New Players; Check In
&lt;/pre&gt;
&lt;/body&gt;
&lt;/html&gt;</string>
</property>
<property name="openExternalLinks">
<bool>true</bool>
</property>
</widget>
</item>
</layout>
</widget>
<resources/>
<connections/>
</ui>

View File

@ -359,7 +359,7 @@
<item row="1" column="2">
<widget class="QPushButton" name="btnTestKey">
<property name="text">
<string>Tesk Key</string>
<string>Test Key</string>
</property>
</widget>
</item>
@ -503,7 +503,7 @@
<enum>QFormLayout::AllNonFixedFieldsGrow</enum>
</property>
<item row="1" column="0">
<widget class="QLabel" name="label">
<widget class="QLabel" name="lblTemplate">
<property name="text">
<string>Template:</string>
</property>
@ -512,31 +512,80 @@
<item row="1" column="1">
<widget class="QLineEdit" name="leRenameTemplate">
<property name="toolTip">
<string>&lt;html&gt;&lt;head/&gt;&lt;body&gt;&lt;p&gt;The template for the new filename. Accepts the following variables:&lt;/p&gt;&lt;p&gt;%series%&lt;br/&gt;%issue%&lt;br/&gt;%volume%&lt;br/&gt;%issuecount%&lt;br/&gt;%year%&lt;br/&gt;%month%&lt;br/&gt;%month_name%&lt;br/&gt;%publisher%&lt;br/&gt;%title%&lt;br/&gt;
%genre%&lt;br/&gt;
%language_code%&lt;br/&gt;
%criticalrating%&lt;br/&gt;
%alternateseries%&lt;br/&gt;
%alternatenumber%&lt;br/&gt;
%alternatecount%&lt;br/&gt;
%imprint%&lt;br/&gt;
%format%&lt;br/&gt;
%maturityrating%&lt;br/&gt;
%storyarc%&lt;br/&gt;
%seriesgroup%&lt;br/&gt;
%scaninfo%
&lt;/p&gt;&lt;p&gt;Examples:&lt;/p&gt;&lt;p&gt;&lt;span style=&quot; font-style:italic;&quot;&gt;%series% %issue% (%year%)&lt;/span&gt;&lt;br/&gt;&lt;span style=&quot; font-style:italic;&quot;&gt;%series% #%issue% - %title%&lt;/span&gt;&lt;/p&gt;&lt;/body&gt;&lt;/html&gt;</string>
<string>&lt;pre&gt;The template for the new filename. Uses python format strings https://docs.python.org/3/library/string.html#format-string-syntax
Accepts the following variables:
{isEmpty} (boolean)
{tagOrigin} (string)
{series} (string)
{issue} (string)
{title} (string)
{publisher} (string)
{month} (integer)
{year} (integer)
{day} (integer)
{issueCount} (integer)
{volume} (integer)
{genre} (string)
{language} (string)
{comments} (string)
{volumeCount} (integer)
{criticalRating} (string)
{country} (string)
{alternateSeries} (string)
{alternateNumber} (string)
{alternateCount} (integer)
{imprint} (string)
{notes} (string)
{webLink} (string)
{format} (string)
{manga} (string)
{blackAndWhite} (boolean)
{pageCount} (integer)
{maturityRating} (string)
{storyArc} (string)
{seriesGroup} (string)
{scanInfo} (string)
{characters} (string)
{teams} (string)
{locations} (string)
{credits} (list of dict({&amp;apos;role&amp;apos;: &amp;apos;str&amp;apos;, &amp;apos;person&amp;apos;: &amp;apos;str&amp;apos;, &amp;apos;primary&amp;apos;: boolean}))
{tags} (list of str)
{pages} (list of dict({&amp;apos;Image&amp;apos;: &amp;apos;str(int)&amp;apos;, &amp;apos;Type&amp;apos;: &amp;apos;str&amp;apos;}))
CoMet-only items:
{price} (float)
{isVersionOf} (string)
{rights} (string)
{identifier} (string)
{lastMark} (string)
{coverImage} (string)
Examples:
{series} {issue} ({year})
Spider-Geddon 1 (2018)
{series} #{issue} - {title}
Spider-Geddon #1 - New Players; Check In
&lt;/pre&gt;</string>
</property>
</widget>
</item>
<item row="2" column="0">
<widget class="QLabel" name="label_6">
<widget class="QPushButton" name="btnTemplateHelp">
<property name="text">
<string>Template Help</string>
</property>
</widget>
</item>
<item row="3" column="0">
<widget class="QLabel" name="lblPadding">
<property name="text">
<string>Issue # Zero Padding</string>
</property>
</widget>
</item>
<item row="2" column="1">
<item row="3" column="1">
<widget class="QLineEdit" name="leIssueNumPadding">
<property name="sizePolicy">
<sizepolicy hsizetype="Maximum" vsizetype="Fixed">
@ -555,7 +604,7 @@
</property>
</widget>
</item>
<item row="3" column="0" colspan="2">
<item row="4" column="0" colspan="2">
<widget class="QCheckBox" name="cbxSmartCleanup">
<property name="toolTip">
<string>&lt;html&gt;&lt;head/&gt;&lt;body&gt;&lt;p&gt;&lt;span style=&quot; font-weight:600;&quot;&gt;&amp;quot;Smart Text Cleanup&amp;quot; &lt;/span&gt;will attempt to clean up the new filename if there are missing fields from the template. For example, removing empty braces, repeated spaces and dashes, and more. Experimental feature.&lt;/p&gt;&lt;/body&gt;&lt;/html&gt;</string>
@ -565,13 +614,33 @@
</property>
</widget>
</item>
<item row="4" column="0" colspan="2">
<item row="5" column="0" colspan="2">
<widget class="QCheckBox" name="cbxChangeExtension">
<property name="text">
<string>Change Extension Based On Archive Type</string>
</property>
</widget>
</item>
<item row="6" column="0">
<widget class="QCheckBox" name="cbxMoveFiles">
<property name="toolTip">
<string>If checked moves files to specified folder</string>
</property>
<property name="text">
<string>Move files when renaming</string>
</property>
</widget>
</item>
<item row="7" column="0">
<widget class="QLabel" name="lblDirectory">
<property name="text">
<string>Destination Directory:</string>
</property>
</widget>
</item>
<item row="7" column="1">
<widget class="QLineEdit" name="leDirectory"/>
</item>
</layout>
</item>
</layout>

View File

@ -512,6 +512,16 @@
</property>
</widget>
</item>
<item row="10" column="1">
<widget class="QLineEdit" name="leSeriesPubYear"/>
</item>
<item row="9" column="1">
<widget class="QLabel" name="label_7">
<property name="text">
<string>Series Year</string>
</property>
</widget>
</item>
</layout>
</item>
<item>

View File

@ -16,9 +16,9 @@
import sys
import platform
import urllib.request, urllib.error, urllib.parse
import requests
import urllib.parse
#import os
#import urllib
try:
from PyQt5.QtNetwork import QNetworkAccessManager, QNetworkRequest, QNetworkReply
@ -47,28 +47,30 @@ class VersionChecker(QObject):
base_url = "http://comictagger1.appspot.com/latest"
args = ""
params = dict()
if use_stats:
params = {
'uuid': uuid,
'version': ctversion.version
}
if platform.system() == "Windows":
plat = "win"
params['platform'] = "win"
elif platform.system() == "Linux":
plat = "lin"
params['platform'] = "lin"
elif platform.system() == "Darwin":
plat = "mac"
params['platform'] = "mac"
else:
plat = "other"
args = "?uuid={0}&platform={1}&version={2}".format(
uuid, plat, ctversion.version)
if not getattr(sys, 'frozen', None):
args += "&src=T"
params['platform'] = "other"
return base_url + args
if not getattr(sys, 'frozen', None):
params['src'] = 'T'
return (base_url, params)
def getLatestVersion(self, uuid, use_stats=True):
try:
resp = urllib.request.urlopen(self.getRequestUrl(uuid, use_stats))
new_version = resp.read()
url, params = self.getRequestUrl(uuid, use_stats)
new_version = requests.get(url, params=params).text
except Exception as e:
return None
@ -79,12 +81,11 @@ class VersionChecker(QObject):
versionRequestComplete = pyqtSignal(str)
def asyncGetLatestVersion(self, uuid, use_stats):
url = self.getRequestUrl(uuid, use_stats)
url, params = self.getRequestUrl(uuid, use_stats)
self.nam = QNetworkAccessManager()
self.nam.finished.connect(self.asyncGetLatestVersionComplete)
self.nam.get(QNetworkRequest(QUrl(str(url))))
self.nam.get(QNetworkRequest(QUrl(str(url + '?' + urllib.parse.urlencode(params)))))
def asyncGetLatestVersionComplete(self, reply):
if (reply.error() != QNetworkReply.NoError):

View File

@ -5,7 +5,7 @@ TAGGER_BASE ?= ../
TAGGER_SRC := $(TAGGER_BASE)/comictaggerlib
APP_NAME := ComicTagger
VERSION_STR := $(shell python setup.py --version)
VERSION_STR := $(shell cd .. && python setup.py --version)
MAC_BASE := $(TAGGER_BASE)/mac
DIST_DIR := $(MAC_BASE)/dist

13
pyproject.toml Normal file
View File

@ -0,0 +1,13 @@
[tool.black]
line-length = 150
[tool.isort]
line_length = 150
[build-system]
requires = ["setuptools>=42", "wheel", "setuptools_scm[toml]>=3.4"]
build-backend = "setuptools.build_meta"
[tool.setuptools_scm]
write_to = "comictaggerlib/ctversion.py"
local_scheme = "no-local-version"

1
requirements-CBR.txt Normal file
View File

@ -0,0 +1 @@
unrar-cffi>=0.2.2

1
requirements-GUI.txt Normal file
View File

@ -0,0 +1 @@
PyQt5<=5.15.3

View File

@ -1,8 +1,7 @@
configparser
beautifulsoup4 >= 4.1
natsort==3.5.2
PyPDF2==1.24
configparser
natsort
pillow>=4.3.0
PyQt5>=5.12.2
pyinstaller>=3.5
unrar-cffi==0.1.0a5
requests
pathvalidate

4
requirements_dev.txt Normal file
View File

@ -0,0 +1,4 @@
pyinstaller==4.3
setuptools>=42
setuptools_scm[toml]>=3.4
wheel

222
setup.py
View File

@ -6,174 +6,76 @@
# It seems that post installation tweaks are broken by wheel files.
# Kept here for further research
from __future__ import print_function
from setuptools import setup
from setuptools import dist
from setuptools import Command
import setuptools.command.build_py
import setuptools.command.install
import subprocess
import glob
import os
import sys
import shutil
import platform
import tempfile
python_requires='>=3',
from setuptools import setup
with open('requirements.txt') as f:
required = f.read().splitlines()
with open('README.md') as f:
long_description = f.read()
def read(fname):
"""
Read the contents of a file.
Parameters
----------
fname : str
Path to file.
Returns
-------
str
File contents.
"""
with open(os.path.join(os.path.dirname(__file__), fname)) as f:
return f.read()
platform_data_files = []
"""
if platform.system() in [ "Windows" ]:
required.append("winshell")
install_requires = read("requirements.txt").splitlines()
# Some files to install on different platforms
# Dynamically determine extra dependencies
extras_require = {}
extra_req_files = glob.glob("requirements-*.txt")
for extra_req_file in extra_req_files:
name = os.path.splitext(extra_req_file)[0].replace("requirements-", "", 1)
extras_require[name] = read(extra_req_file).splitlines()
if platform.system() == "Linux":
linux_desktop_shortcut = "/usr/local/share/applications/ComicTagger.desktop"
platform_data_files = [("/usr/local/share/applications",
["desktop-integration/linux/ComicTagger.desktop"]),
("/usr/local/share/comictagger",
["comictaggerlib/graphics/app.png"]),
]
if platform.system() == "Windows":
win_desktop_folder = os.path.join(os.environ["USERPROFILE"], "Desktop")
win_appdata_folder = os.path.join(os.environ["APPDATA"], "comictagger")
win_desktop_shortcut = os.path.join(win_desktop_folder, "ComicTagger-pip.lnk")
platform_data_files = [(win_desktop_folder,
["desktop-integration/windows/ComicTagger-pip.lnk"]),
(win_appdata_folder,
["windows/app.ico"]),
]
# If there are any extras, add a catch-all case that includes everything.
# This assumes that entries in extras_require are lists (not single strings),
# and that there are no duplicated packages across the extras.
if extras_require:
extras_require["all"] = sorted({x for v in extras_require.values() for x in v})
if platform.system() == "Darwin":
mac_app_folder = "/Applications"
ct_app_name = "ComicTagger-pip.app"
mac_app_infoplist = os.path.join(mac_app_folder, ct_app_name, "Contents", "Info.plist")
mac_app_main = os.path.join(mac_app_folder, ct_app_name, "MacOS", "main.sh")
mac_python_link = os.path.join(mac_app_folder, ct_app_name, "MacOS", "ComicTagger")
platform_data_files = [(os.path.join(mac_app_folder, ct_app_name, "Contents"),
["desktop-integration/mac/Info.plist"]),
(os.path.join(mac_app_folder, ct_app_name, "Contents/Resources"),
["mac/app.icns"]),
(os.path.join(mac_app_folder, ct_app_name, "Contents/MacOS"),
["desktop-integration/mac/main.sh",
"desktop-integration/mac/ComicTagger"]),
]
def fileTokenReplace(filename, token, replacement):
with open(filename, "rt") as fin:
fd, tmpfile = tempfile.mkstemp()
with open(tmpfile, "wt") as fout:
for line in fin:
fout.write(line.replace('%%{}%%'.format(token), replacement))
os.close(fd)
# fix permissions of temp file
os.chmod(tmpfile, 420) #Octal 0o644
os.rename(tmpfile, filename)
def postInstall(scripts_folder):
entry_point_script = os.path.join(scripts_folder, "comictagger")
if platform.system() == "Windows":
# doctor the shortcut for this windows system after deployment
import winshell
winshell.CreateShortcut(
Path=os.path.abspath(win_desktop_shortcut),
Target=entry_point_script + ".exe",
Icon=(os.path.join(win_appdata_folder, 'app.ico'), 0),
Description="Launch ComicTagger as installed by PIP"
)
if platform.system() == "Linux":
# doctor the script path in the desktop file
fileTokenReplace(linux_desktop_shortcut,
"CTSCRIPT",
entry_point_script)
if platform.system() == "Darwin":
# doctor the plist app version
fileTokenReplace(mac_app_infoplist,
"CTVERSION",
comictaggerlib.ctversion.version)
# doctor the script path in main.sh
fileTokenReplace(mac_app_main,
"CTSCRIPT",
entry_point_script)
# Make the launcher script executable
os.chmod(mac_app_main, 509) #Octal 0o775
# Final install step: create a symlink to Python OS X application
punt = False
pythonpath,top = os.path.split(os.path.realpath(sys.executable))
while top:
if 'Resources' in pythonpath:
pass
elif os.path.exists(os.path.join(pythonpath,'Resources')):
break
pythonpath,top = os.path.split(pythonpath)
else:
print("Failed to find a Resources directory associated with ", str(sys.executable))
punt = True
if not punt:
pythonapp = os.path.join(pythonpath, 'Resources','Python.app','Contents','MacOS','Python')
if not os.path.exists(pythonapp):
print("Failed to find a Python app in ", str(pythonapp))
punt = True
# remove the placeholder
os.remove(mac_python_link)
if not punt:
os.symlink(pythonapp, mac_python_link)
else:
# We failed, but we can still be functional
os.symlink(sys.executable, mac_python_link)
"""
setup(name="comictagger",
install_requires=required,
description="A cross-platform GUI/CLI app for writing metadata to comic archives",
author="ComicTagger team",
author_email="comictagger@gmail.com",
url="https://github.com/comictagger/comictagger",
packages=["comictaggerlib", "comicapi"],
package_data={
'comictaggerlib': ['ui/*', 'graphics/*'],
},
entry_points=dict(console_scripts=['comictagger=comictaggerlib.main:ctmain']),
data_files=platform_data_files,
setup_requires=[
"setuptools_scm"
],
use_scm_version={
'write_to': 'comictaggerlib/ctversion.py'
},
classifiers=[
"Development Status :: 4 - Beta",
"Environment :: Console",
"Environment :: Win32 (MS Windows)",
"Environment :: MacOS X",
"Environment :: X11 Applications :: Qt",
"Intended Audience :: End Users/Desktop",
"License :: OSI Approved :: Apache Software License",
"Natural Language :: English",
"Operating System :: OS Independent",
"Programming Language :: Python :: 3.5",
"Programming Language :: Python :: 3.6",
"Topic :: Utilities",
"Topic :: Other/Nonlisted Topic",
"Topic :: Multimedia :: Graphics"
],
keywords=['comictagger', 'comics', 'comic', 'metadata', 'tagging', 'tagger'],
license="Apache License 2.0",
long_description=long_description,
long_description_content_type="text/markdown"
setup(
name="comictagger",
install_requires=install_requires,
extras_require=extras_require,
python_requires=">=3",
description="A cross-platform GUI/CLI app for writing metadata to comic archives",
author="ComicTagger team",
author_email="comictagger@gmail.com",
url="https://github.com/comictagger/comictagger",
packages=["comictaggerlib", "comicapi"],
package_data={
"comictaggerlib": ["ui/*", "graphics/*"],
},
entry_points=dict(console_scripts=["comictagger=comictaggerlib.main:ctmain"]),
classifiers=[
"Development Status :: 4 - Beta",
"Environment :: Console",
"Environment :: Win32 (MS Windows)",
"Environment :: MacOS X",
"Environment :: X11 Applications :: Qt",
"Intended Audience :: End Users/Desktop",
"License :: OSI Approved :: Apache Software License",
"Natural Language :: English",
"Operating System :: OS Independent",
"Programming Language :: Python :: 3.5",
"Programming Language :: Python :: 3.6",
"Topic :: Utilities",
"Topic :: Other/Nonlisted Topic",
"Topic :: Multimedia :: Graphics",
],
keywords=["comictagger", "comics", "comic", "metadata", "tagging", "tagger"],
license="Apache License 2.0",
long_description=read("README.md"),
long_description_content_type='text/markdown'
)