First commit of tdl

Initial round of re-branding, I temporarily am using the dta icon just
rotated 180, waiting for a permanent icon to use
Removed the initial create tab for dta but left any download links as is
Added webextension-polyfill-ts to enable retrieval of cookies
Removed all access and preferences to the download manager as it is not
relevant in this fork

.github/ISSUE_TEMPLATE/bug_report.md

@@ -1,6 +1,6 @@
 ---
 name: Bug report
-about: Create a report to help us improve
+about: Create a report to help us improve DownThemAll!
 title: ''
 labels: ''
 assignees: ''
@@ -8,9 +8,9 @@ assignees: ''
 ---
 
 **Desktop (please complete the following information):**
- - OS: [e.g. iOS]
- - Browser [e.g. chrome, safari]
- - Version [e.g. 22]
+ - OS: [e.g. Windows 10, macOS, Linux (distribution, desktop environment)]
+ - Browser and version: [e.g. Firefox 42, Chrome 70, Opera 15, Seamonkey 2.16]
+ - DownThemAll! version: [e.g. 4.2, 3.0, latest]
 
 **Describe the bug**
 A clear and concise description of what the bug is.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -1,6 +1,6 @@
 ---
 name: Feature request
-about: Suggest an idea for this project
+about: Suggest a feature or an idea for DownThemAll!
 title: ''
 labels: ''
 assignees: ''

LICENSE.md

@@ -77,3 +77,13 @@ Licensed under the Mozilla Public License 2.0.
 
 The list itself is licensed under the Mozilla Public License 2.0.
 The javascript library accessing it is licensed under the MIT license.
+
+## whatwg-mimetype
+
+Licensed under MIT
+
+
+## CDHeaderParser
+
+Licensed under MPL-2
+(c) 2017 Rob Wu <rob@robwu.nl> (https://robwu.nl)

Readme.md

@@ -1,12 +1,14 @@
-DownThemAll! WE
-===
+
+![DownThemAll!](https://raw.githubusercontent.com/downthemall/downthemall/master/style/icon128.png)
+
+
+# DownThemAll! WE
 
 The DownThemAll! WebExtension.
 
 For those still on supported browser: [Non-WebExtension legacy code](https://github.com/downthemall/downthemall-legacy).
 
-About
----
+## About
 
 This is the WebExtension version of DownThemAll!, a complete re-development from scratch.
 Being a WebExtension it lacks a ton of features the original DownThemAll! had. Sorry, but there is no way around it since Mozilla decided to adopt WebExtensions as the *only* extension type and WebExtensions are extremely limited in what they can do.
@@ -23,25 +25,79 @@ But it is what it is...
 
 **What we *can* do and did do is bring the mass selection, organizing (renaming masks, etc) and queueing tools of DownThemAll! over to the WebExtension, so you can easily queue up hundreds or thousands files at once without the downloads going up in flames because the browser tried to download them all at once.**
 
-Translations
----
+## Translations
 
 If you would like to help out translating DTA, please see our [translation guide](_locales/Readme.md).
 
-Development
----
+## Development
 
-You will want to `yarn` the development dependencies such as webpack first.
+### Requirements
 
-Afterwards there is two important commands to run
+ - [node](https://nodejs.org/en/)
+ - [yarn](https://yarnpkg.com/)
+ - [python3](https://www.python.org/) >= 3.6 (to build zips)
+ - [web-ext](https://www.npmjs.com/package/web-ext) (for development ease)
  
-  * `yarn watch` - This will run the webpack bundler in watch mode, updating bundles as you change the source.
-  * `yarn webext` - This will run the WebExtension in a development profile using the [`web-ext` tool from mozilla](https://www.npmjs.com/package/web-ext) (which you need to install separately). This will use the directory `../dtalite.p` to keep a development profile. You might need to create this directory before you use this command first.
+### Setup
 
-Please note: You have to run `yarn watch` (at least once) as it builds the actual script bundles.
+You will want to run `yarn` to install the development dependencies such as webpack first.
 
-Alternative, you can also `yarn build`, which then builds an *unsigned* zip that you can then install permanently in a browser that does not enforce signing (i.e. Nightly or the Unbranded Firefox).
+### Making changes
 
-Before submitting patches, please make sure you run eslint, if this isn't done automatically, and eslint does not report any open issues. Code contributions should favor typescript code over javascript code. External dependencies that would ship with the final product (including all npm/yarn packages) should be kept to a bare minimum.
+Just use your favorite text editor to edit the files.
 
-The code base is comparatively large for a WebExtension, with over 10K sloc of typescript and over 14K sloc total.
+You will want to run`yarn watch`.
+This will run the webpack bundler in watch mode, transpiling the TypeScript to Javascript and updating bundles as you change the source.
+
+Please note: You have to run `yarn watch` or `yarn build` (at least once) as it builds the actual script bundles.
+
+### Running in Firefox
+
+I recommend you install the [`web-ext`](https://www.npmjs.com/package/web-ext) tools from mozilla. It is not listed as a dependency by design at it causes problems with dependency resolution in yarn right now if installed in the same location as the rest of the dependencies.
+
+If you did, then running `yarn webext` (additionally to `yarn watch`) will run the WebExtension in a development profile. This will use the directory `../dtalite.p` to keep a development profile. You might need to create this directory before you use this command. Furthermore `yarn webext` will watch for changes to the sources and automatically reload the extension.
+  
+Alternatively, you can also `yarn build`, which then builds an *unsigned* zip that you can then install permanently in a browser that does not enforce signing (i.e. Nightly or the Unbranded Firefox with the right about:config preferences).
+
+### Running in Chrome/Chromium/etc
+
+You have to build the bundles first, of course.
+
+Then put your Chrome into Developement Mode on the Extensions page, and Load Unpacked the directory of your downthemall clone.
+
+### Making release zips
+
+To get a basic unofficial set of zips for Firefox and chrome, run `yarn build`.
+
+If you want to generate release builds like the ones that are eventually released in the extension stores, use `python3 util/build.py --mode=release`.
+
+The output is located in `web-ext-artifacts`.
+
+ - `-fx.zip` are Firefox builds
+ - `-crx.zip` are Chrome/Chromium builds
+ - `-opr.zip` are Opera builds (essentially like the Chrome one, but without sounds)
+
+### The AMO Editors tl;dr guide
+
+  1. Install the requirements.
+  2. `yarn && python3 build/util.py --mode=release`
+  3. Have a look in `web-ext-artifacts/dta-*-fx.zip`
+
+
+### Patches
+
+Before submitting patches, please make sure you run eslint (if this isn't done automatically in your text editor/IDE), and eslint does not report any open issues. Code contributions should favor typescript code over javascript code. External dependencies that would ship with the final product (including all npm/yarn packages) should be kept to a bare minimum and need justification.
+
+Please submit your patches as Pull Requests, and rebase your commits onto the current `master` before submitting.
+
+### Code structure
+
+The code base is comparatively large for a WebExtension, with over 11K sloc of typescript.
+It isn't as well organized as it should be in some places; hope you don't mind.
+
+* `uikit/` - The base User Interface Kit, which currently consists of
+  * the `VirtualTable` implementation, aka that interactive HTML table with columns, columns resizing and hiding, etc you see in the Manager, Select and Preferences windows/tabs
+  * the `ContextMenu` and related classes that drive the HTML-based context menus
+* `lib/` - The "backend stuff" and assorted library routines and classes.
+* `windows/` - The "frontend stuff" so all the HTML and corresponding code to make that HTML into something interactive
+* `style/` - CSS and images

TODO.md

@@ -1,19 +1,13 @@
 TODO
 ---
 
-aka a lot
-
 P2
 ===
 
 Planned for later.
 
-* Soft errors and retry logic
-  * Big caveat: When the server still responds, like 50x errors which would be recoverable, we actually have no way of knowing it did in respond in such a way. See P4 - Handle Errors remarks.
-* Delete files (well, as far as the browser allows)
 * Inter-addon API (basic)
   * Add downloads
-* Chrome support
 * vtable perf: cache column widths
 * Download options
   * This is a bit more limited, as we cannot modify options of downloads that have been started (and paused) or that are done.
@@ -30,12 +24,7 @@ Nice-to-haves.
   * Manipulate downloads (e.g. rewrite URLs)
 * Native context menus?
   * Would require massive reworks incl the need for new icon formats, but potentially feasible.
-* Import/Export
 * Download priorities (manual scheduling overrides)
-* Dark Theme support
-  * os/browser define be default
-  * overwritable
-* Get and cache system icons (because Firefox doesn't allow moz-icon: for WE, but makes them kinda accessible through the downloads API anyway, essentially copying them via a canvas on a privileged hidden page into a data URL... ikr)
 * Remove `any` types as possible, and generally improve typescript (new language to me)
 
 P4
@@ -47,8 +36,6 @@ Stuff that probably cannot be implemented due to WeberEension limitations.
   * Firefox helpfully keeps different lists of downloads. One for newly added downloads, and other ones for "previous" downloads. Turns out the WebExtension API only ever queries the "new" list.
 * Segmented downloads
   * Cannot be done with WebExtensions - downloads API has no support and manually downloading, storing in temporary add-on storage and reassmbling the downloaded parts later is not only efficient but does not reliabliy work due to storage limitations.
-* Handle errors, 404 and such
-  * The Firefox download manager is too stupid and webRequest does not see Downloads, so cannot be done right now.
 * Conflicts: ask when a file exists
   * Not supported by Firefox
 * Speed limiter

_locales/all.json

@@ -0,0 +1,26 @@
+{
+  "ar": "العربية [ar]",
+  "bg": "Български [bg]",
+  "cs": "Čeština (CZ) [cs]",
+  "da": "Dansk [da]",
+  "de": "Deutsch [de]",
+  "el": "Ελληνικά [el]",
+  "en": "English (US) [en]",
+  "es": "Español (España) [es]",
+  "et": "Eesti keel [et]",
+  "fr": "Français [fr]",
+  "hu": "Magyar (HU) [hu]",
+  "id": "Bahasa Indonesia [id]",
+  "it": "Italiano [it]",
+  "ja": "日本語 (JP) [ja]",
+  "ko": "한국어 [ko]",
+  "lt": "Lietuvių [lt]",
+  "nl": "Nederlands [nl]",
+  "pl": "Polski [pl]",
+  "pt": "Português (Brasil) [pt]",
+  "ru": "Русский [ru]",
+  "sv": "Svenska (SV) [sv]",
+  "tr": "Türkçe TR) [tr]",
+  "zh_CN": "中文（简体） [zh_CN]",
+  "zh_TW": "正體中文 (TW) [zh_TW]"
+}

data/filters.json

@@ -15,7 +15,7 @@
 	"deffilter-aud": {
 		"label": "Audio",
 		"expr": "/\\.(?:mp3|wav|og(?:g|a)|flac|midi?|rm|aac|wma|mka|ape|opus)$/i",
-		"type": 1,
+		"type": 3,
 		"active": false,
 		"icon": "mp3"
 	},

data/mime.json

@@ -0,0 +1,396 @@
+{
+  "e": {
+    "3gpp": "3gp",
+    "asx": "asf",
+    "gz": "gzip",
+    "heic": "heif",
+    "html": [
+      "htm",
+      "shtml",
+      "php"
+    ],
+    "jar": [
+      "war",
+      "ear"
+    ],
+    "jpg": [
+      "jpeg",
+      "jpe",
+      "jfif"
+    ],
+    "js": "jsx",
+    "mid": [
+      "midi",
+      "kar"
+    ],
+    "mkv": [
+      "mk3d",
+      "mks"
+    ],
+    "mov": [
+      "qt",
+      "moov"
+    ],
+    "mpg": [
+      "mpe",
+      "mpeg"
+    ],
+    "pem": [
+      "crt",
+      "der"
+    ],
+    "pl": "pm",
+    "prc": "pdb",
+    "ps": [
+      "eps",
+      "ai"
+    ],
+    "svg": "svgz",
+    "tcl": "tk",
+    "tif": "tiff"
+  },
+  "m": {
+    "application/7z": "7z",
+    "application/7z-compressed": "7z",
+    "application/ai": "ps",
+    "application/atom": "atom",
+    "application/atom+xml": "atom",
+    "application/bz2": "bz2",
+    "application/bzip2": "bz2",
+    "application/cco": "cco",
+    "application/cocoa": "cco",
+    "application/compressed": "gz",
+    "application/crt": "pem",
+    "application/der": "pem",
+    "application/doc": "doc",
+    "application/ear": "jar",
+    "application/eot": "eot",
+    "application/eps": "ps",
+    "application/gz": "gz",
+    "application/gzip": "gz",
+    "application/hqx": "hqx",
+    "application/jar": "jar",
+    "application/jardiff": "jardiff",
+    "application/java-archive": "jar",
+    "application/java-archive-diff": "jardiff",
+    "application/java-jnlp-file": "jnlp",
+    "application/javascript": "js",
+    "application/jnlp": "jnlp",
+    "application/js": "js",
+    "application/json": "json",
+    "application/jsx": "js",
+    "application/kml": "kml",
+    "application/kmz": "kmz",
+    "application/m3u8": "m3u8",
+    "application/mac-binhex40": "hqx",
+    "application/makeself": "run",
+    "application/msword": "doc",
+    "application/odg": "odg",
+    "application/odp": "odp",
+    "application/ods": "ods",
+    "application/odt": "odt",
+    "application/pdb": "prc",
+    "application/pdf": "pdf",
+    "application/pem": "pem",
+    "application/perl": "pl",
+    "application/pilot": "prc",
+    "application/pl": "pl",
+    "application/pm": "pl",
+    "application/postscript": "ps",
+    "application/ppt": "ppt",
+    "application/prc": "prc",
+    "application/ps": "ps",
+    "application/rar": "rar",
+    "application/rar-compressed": "rar",
+    "application/redhat-package-manager": "rpm",
+    "application/rpm": "rpm",
+    "application/rss": "rss",
+    "application/rss+xml": "rss",
+    "application/rtf": "rtf",
+    "application/run": "run",
+    "application/sea": "sea",
+    "application/shockwave-flash": "swf",
+    "application/sit": "sit",
+    "application/stuffit": "sit",
+    "application/swf": "swf",
+    "application/tar": "tar",
+    "application/tcl": "tcl",
+    "application/tk": "tcl",
+    "application/vnd.apple.mpegurl": "m3u8",
+    "application/vnd.google-earth.kml+xml": "kml",
+    "application/vnd.google-earth.kmz": "kmz",
+    "application/vnd.ms-excel": "xls",
+    "application/vnd.ms-fontobject": "eot",
+    "application/vnd.ms-powerpoint": "ppt",
+    "application/vnd.oasis.opendocument.graphics": "odg",
+    "application/vnd.oasis.opendocument.presentation": "odp",
+    "application/vnd.oasis.opendocument.spreadsheet": "ods",
+    "application/vnd.oasis.opendocument.text": "odt",
+    "application/vnd.wap.wmlc": "wmlc",
+    "application/war": "jar",
+    "application/wmlc": "wmlc",
+    "application/x-7z": "7z",
+    "application/x-7z-compressed": "7z",
+    "application/x-ai": "ps",
+    "application/x-atom": "atom",
+    "application/x-atom+xml": "atom",
+    "application/x-bz2": "bz2",
+    "application/x-bzip2": "bz2",
+    "application/x-cco": "cco",
+    "application/x-cocoa": "cco",
+    "application/x-compressed": "gz",
+    "application/x-crt": "pem",
+    "application/x-der": "pem",
+    "application/x-doc": "doc",
+    "application/x-ear": "jar",
+    "application/x-eot": "eot",
+    "application/x-eps": "ps",
+    "application/x-gz": "gz",
+    "application/x-gzip": "gz",
+    "application/x-hqx": "hqx",
+    "application/x-jar": "jar",
+    "application/x-jardiff": "jardiff",
+    "application/x-java-archive": "jar",
+    "application/x-java-archive-diff": "jardiff",
+    "application/x-java-jnlp-file": "jnlp",
+    "application/x-javascript": "js",
+    "application/x-jnlp": "jnlp",
+    "application/x-js": "js",
+    "application/x-json": "json",
+    "application/x-jsx": "js",
+    "application/x-kml": "kml",
+    "application/x-kmz": "kmz",
+    "application/x-m3u8": "m3u8",
+    "application/x-mac-binhex40": "hqx",
+    "application/x-makeself": "run",
+    "application/x-msword": "doc",
+    "application/x-odg": "odg",
+    "application/x-odp": "odp",
+    "application/x-ods": "ods",
+    "application/x-odt": "odt",
+    "application/x-pdb": "prc",
+    "application/x-pdf": "pdf",
+    "application/x-pem": "pem",
+    "application/x-perl": "pl",
+    "application/x-pilot": "prc",
+    "application/x-pl": "pl",
+    "application/x-pm": "pl",
+    "application/x-postscript": "ps",
+    "application/x-ppt": "ppt",
+    "application/x-prc": "prc",
+    "application/x-ps": "ps",
+    "application/x-rar": "rar",
+    "application/x-rar-compressed": "rar",
+    "application/x-redhat-package-manager": "rpm",
+    "application/x-rpm": "rpm",
+    "application/x-rss": "rss",
+    "application/x-rss+xml": "rss",
+    "application/x-rtf": "rtf",
+    "application/x-run": "run",
+    "application/x-sea": "sea",
+    "application/x-shockwave-flash": "swf",
+    "application/x-sit": "sit",
+    "application/x-stuffit": "sit",
+    "application/x-swf": "swf",
+    "application/x-tar": "tar",
+    "application/x-tcl": "tcl",
+    "application/x-tk": "tcl",
+    "application/x-vnd.apple.mpegurl": "m3u8",
+    "application/x-vnd.google-earth.kml+xml": "kml",
+    "application/x-vnd.google-earth.kmz": "kmz",
+    "application/x-vnd.ms-excel": "xls",
+    "application/x-vnd.ms-fontobject": "eot",
+    "application/x-vnd.ms-powerpoint": "ppt",
+    "application/x-vnd.oasis.opendocument.graphics": "odg",
+    "application/x-vnd.oasis.opendocument.presentation": "odp",
+    "application/x-vnd.oasis.opendocument.spreadsheet": "ods",
+    "application/x-vnd.oasis.opendocument.text": "odt",
+    "application/x-vnd.wap.wmlc": "wmlc",
+    "application/x-war": "jar",
+    "application/x-wmlc": "wmlc",
+    "application/x-x509-ca-cert": "pem",
+    "application/x-xhtml": "xhtml",
+    "application/x-xhtml+xml": "xhtml",
+    "application/x-xls": "xls",
+    "application/x-xpi": "xpi",
+    "application/x-xpinstall": "xpi",
+    "application/x-xspf": "xspf",
+    "application/x-xspf+xml": "xspf",
+    "application/x-xz": "xz",
+    "application/x-zip": "zip",
+    "application/x509-ca-cert": "pem",
+    "application/xhtml": "xhtml",
+    "application/xhtml+xml": "xhtml",
+    "application/xls": "xls",
+    "application/xpi": "xpi",
+    "application/xpinstall": "xpi",
+    "application/xspf": "xspf",
+    "application/xspf+xml": "xspf",
+    "application/xz": "xz",
+    "application/zip": "zip",
+    "audio/kar": "mid",
+    "audio/m4a": "m4a",
+    "audio/matroska": "mka",
+    "audio/mid": "mid",
+    "audio/midi": "mid",
+    "audio/mka": "mka",
+    "audio/mp3": "mp3",
+    "audio/mpeg": "mp3",
+    "audio/ogg": "ogg",
+    "audio/ra": "ra",
+    "audio/realaudio": "ra",
+    "audio/x-kar": "mid",
+    "audio/x-m4a": "m4a",
+    "audio/x-matroska": "mka",
+    "audio/x-mid": "mid",
+    "audio/x-midi": "mid",
+    "audio/x-mka": "mka",
+    "audio/x-mp3": "mp3",
+    "audio/x-mpeg": "mp3",
+    "audio/x-ogg": "ogg",
+    "audio/x-ra": "ra",
+    "audio/x-realaudio": "ra",
+    "font/woff": "woff",
+    "font/woff2": "woff2",
+    "font/x-woff": "woff",
+    "font/x-woff2": "woff2",
+    "image/bmp": "bmp",
+    "image/gif": "gif",
+    "image/heic": "heic",
+    "image/heif": "heic",
+    "image/heif-sequence": "heic",
+    "image/ico": "ico",
+    "image/icon": "ico",
+    "image/jfif": "jpg",
+    "image/jng": "jng",
+    "image/jpe": "jpg",
+    "image/jpeg": "jpg",
+    "image/jpg": "jpg",
+    "image/ms-bmp": "bmp",
+    "image/png": "png",
+    "image/svg": "svg",
+    "image/svg+xml": "svg",
+    "image/svgz": "svg",
+    "image/tif": "tif",
+    "image/tiff": "tif",
+    "image/vnd.wap.wbmp": "wbmp",
+    "image/wbmp": "wbmp",
+    "image/webp": "webp",
+    "image/x-bmp": "bmp",
+    "image/x-gif": "gif",
+    "image/x-heic": "heic",
+    "image/x-heif": "heic",
+    "image/x-heif-sequence": "heic",
+    "image/x-ico": "ico",
+    "image/x-icon": "ico",
+    "image/x-jfif": "jpg",
+    "image/x-jng": "jng",
+    "image/x-jpe": "jpg",
+    "image/x-jpeg": "jpg",
+    "image/x-jpg": "jpg",
+    "image/x-ms-bmp": "bmp",
+    "image/x-png": "png",
+    "image/x-svg": "svg",
+    "image/x-svg+xml": "svg",
+    "image/x-svgz": "svg",
+    "image/x-tif": "tif",
+    "image/x-tiff": "tif",
+    "image/x-vnd.wap.wbmp": "wbmp",
+    "image/x-wbmp": "wbmp",
+    "image/x-webp": "webp",
+    "text/component": "htc",
+    "text/css": "css",
+    "text/htc": "htc",
+    "text/htm": "html",
+    "text/html": "html",
+    "text/jad": "jad",
+    "text/javascript": "js",
+    "text/js": "js",
+    "text/jsx": "js",
+    "text/mathml": "mml",
+    "text/mml": "mml",
+    "text/php": "html",
+    "text/plain": "txt",
+    "text/shtml": "html",
+    "text/txt": "txt",
+    "text/vnd.sun.j2me.app-descriptor": "jad",
+    "text/vnd.wap.wml": "wml",
+    "text/wml": "wml",
+    "text/x-component": "htc",
+    "text/x-css": "css",
+    "text/x-htc": "htc",
+    "text/x-htm": "html",
+    "text/x-html": "html",
+    "text/x-jad": "jad",
+    "text/x-javascript": "js",
+    "text/x-js": "js",
+    "text/x-jsx": "js",
+    "text/x-mathml": "mml",
+    "text/x-mml": "mml",
+    "text/x-php": "html",
+    "text/x-plain": "txt",
+    "text/x-shtml": "html",
+    "text/x-txt": "txt",
+    "text/x-vnd.sun.j2me.app-descriptor": "jad",
+    "text/x-vnd.wap.wml": "wml",
+    "text/x-wml": "wml",
+    "text/x-xml": "xml",
+    "text/xml": "xml",
+    "video/3gp": "3gpp",
+    "video/3gpp": "3gpp",
+    "video/asf": "asx",
+    "video/asx": "asx",
+    "video/avi": "avi",
+    "video/flv": "flv",
+    "video/m4v": "m4v",
+    "video/matroska": "mkv",
+    "video/mk3d": "mkv",
+    "video/mks": "mkv",
+    "video/mkv": "mkv",
+    "video/mng": "mng",
+    "video/moov": "mov",
+    "video/mov": "mov",
+    "video/mp2t": "ts",
+    "video/mp4": "mp4",
+    "video/mpe": "mpg",
+    "video/mpeg": "mpg",
+    "video/mpg": "mpg",
+    "video/ms-asf": "asx",
+    "video/ms-wmv": "wmv",
+    "video/msvideo": "avi",
+    "video/opus": "opus",
+    "video/qt": "mov",
+    "video/quicktime": "mov",
+    "video/ts": "ts",
+    "video/webm": "webm",
+    "video/wmv": "wmv",
+    "video/x-3gp": "3gpp",
+    "video/x-3gpp": "3gpp",
+    "video/x-asf": "asx",
+    "video/x-asx": "asx",
+    "video/x-avi": "avi",
+    "video/x-flv": "flv",
+    "video/x-m4v": "m4v",
+    "video/x-matroska": "mkv",
+    "video/x-mk3d": "mkv",
+    "video/x-mks": "mkv",
+    "video/x-mkv": "mkv",
+    "video/x-mng": "mng",
+    "video/x-moov": "mov",
+    "video/x-mov": "mov",
+    "video/x-mp2t": "ts",
+    "video/x-mp4": "mp4",
+    "video/x-mpe": "mpg",
+    "video/x-mpeg": "mpg",
+    "video/x-mpg": "mpg",
+    "video/x-ms-asf": "asx",
+    "video/x-ms-wmv": "wmv",
+    "video/x-msvideo": "avi",
+    "video/x-opus": "opus",
+    "video/x-qt": "mov",
+    "video/x-quicktime": "mov",
+    "video/x-ts": "ts",
+    "video/x-webm": "webm",
+    "video/x-wmv": "wmv"
+  }
+}

data/prefs.json

@@ -1,8 +1,10 @@
 {
-	"global-turbo": false,
+	"button-type": "popup",
+	"manager-in-popup": false,
 	"concurrent": 4,
 	"queue-notification": true,
 	"finish-notification": true,
+	"sounds": true,
 	"open-manager-on-queue": true,
 	"text-links": true,
 	"add-paused": false,
@@ -13,6 +15,9 @@
 	"tooltip": true,
 	"show-urls": false,
 	"remove-missing-on-init": false,
+	"retries": 5,
+	"retry-time": 10,
+	"theme": "default",
 	"limits": [
 		{
 			"domain": "*",

lib/api.ts

@@ -11,7 +11,7 @@ import { getManager } from "./manager/man";
 import { select } from "./select";
 import { single } from "./single";
 import { Notification } from "./notifications";
-import { MASK, FASTFILTER } from "./recentlist";
+import { MASK, FASTFILTER, SUBFOLDER, SERVER } from "./recentlist";
 import { openManager } from "./windowutils";
 import { _ } from "./i18n";
 
@@ -19,7 +19,10 @@ const MAX_BATCH = 10000;
 
 export interface QueueOptions {
   mask?: string;
+  subfolder?: string;
+  server?: string;
   paused?: boolean;
+  cookies?: boolean;
 }
 
 export const API = new class APIImpl {
@@ -28,10 +31,13 @@ export const API = new class APIImpl {
   }
 
   async queue(items: BaseItem[], options: QueueOptions) {
-    await MASK.init();
+    await Promise.all([MASK.init(), SUBFOLDER.init()]);
     const {mask = MASK.current} = options;
+    const {subfolder = SUBFOLDER.current} = options;
+    const {server = SERVER.current} = options;
 
     const {paused = false} = options;
+    const {cookies = false} = options;
     const defaults: any = {
       _idx: 0,
       get idx() {
@@ -46,8 +52,11 @@ export const API = new class APIImpl {
       private: false,
       postData: null,
       mask,
+      subfolder,
+      server,
       date: Date.now(),
-      paused
+      paused,
+      cookies,
     };
     let currentBatch = await Prefs.get("currentBatch", 0);
     const initialBatch = currentBatch;
@@ -77,7 +86,7 @@ export const API = new class APIImpl {
         new Notification(null, _("queued-downloads", items.length));
       }
     }
-    if (await Prefs.get("open-manager-on-queue")) {
+    if (false && await Prefs.get("open-manager-on-queue")) {
       await openManager(false);
     }
   }
@@ -117,6 +126,14 @@ export const API = new class APIImpl {
       await FASTFILTER.init();
       await FASTFILTER.push(options.fast);
     }
+    if (typeof options.subfolder === "string" && !options.subfolderOnce) {
+      await SUBFOLDER.init();
+      await SUBFOLDER.push(options.subfolder);
+    }
+    if (typeof options.server === "string" && !options.serverOnce) {
+      await SERVER.init();
+      await SERVER.push(options.server);
+    }
     if (typeof options.type === "string") {
       await Prefs.set("last-type", options.type);
     }

lib/background.ts

@@ -18,18 +18,42 @@ import {
   Tab,
   // eslint-disable-next-line no-unused-vars
   MenuClickInfo,
+  CHROME,
+  runtime,
+  history,
+  sessions,
+  // eslint-disable-next-line no-unused-vars
+  OnInstalled,
 } from "./browser";
 import { Bus } from "./bus";
 import { filterInSitu } from "./util";
+import { DB } from "./db";
 
 
 const menus = typeof (_menus) !== "undefined" && _menus || _cmenus;
 
 const GATHER = "/bundles/content-gather.js";
 
+const CHROME_CONTEXTS = Object.freeze(new Set([
+  "all",
+  "audio",
+  "browser_action",
+  "editable",
+  "frame",
+  "image",
+  "launcher",
+  "link",
+  "page",
+  "page_action",
+  "selection",
+  "video",
+]));
 
 async function runContentJob(tab: Tab, file: string, msg: any) {
   try {
+    if (tab && tab.incognito && msg) {
+      msg.private = tab.incognito;
+    }
     const res = await tabs.executeScript(tab.id, {
       file,
       allFrames: true,
@@ -83,15 +107,19 @@ class Handler {
 
   async performSelection(options: SelectionOptions) {
     try {
-      const selectedTabs = options.allTabs ?
-        await tabs.query({
+      const tabOptions: any = {
         currentWindow: true,
         discarded: false,
-          hidden: false}) as any[] :
+      };
+      if (!CHROME) {
+        tabOptions.hidden = false;
+      }
+      const selectedTabs = options.allTabs ?
+        await tabs.query(tabOptions) as any[] :
         [options.tab];
 
       const textLinks = await Prefs.get("text-links", true);
-      const goptions = {
+      const gatherOptions = {
         type: "DTA:gather",
         selectionOnly: options.selectionOnly,
         textLinks,
@@ -100,7 +128,7 @@ class Handler {
       };
 
       const results = await Promise.all(selectedTabs.
-        map((tab: any) => runContentJob(tab, GATHER, goptions)));
+        map((tab: any) => runContentJob(tab, GATHER, gatherOptions)));
 
       await this.processResults(options.turbo, results.flat());
     }
@@ -110,48 +138,54 @@ class Handler {
   }
 }
 
+function getMajor(version?: string) {
+  if (!version) {
+    return "";
+  }
+  const match = version.match(/^\d+\.\d+/);
+  if (!match) {
+    return "";
+  }
+  return match[0];
+}
+
+runtime.onInstalled.addListener(({reason, previousVersion}: OnInstalled) => {
+  const {version} = runtime.getManifest();
+  const major = getMajor(version);
+  const prevMajor = getMajor(previousVersion);
+  if (reason === "update" && major !== prevMajor) {
+    // tabs.create({
+    //   url: `https://about.downthemall.org/changelog/?cur=${major}&prev=${prevMajor}`,
+    // });
+  }
+  else if (reason === "install") {
+    // tabs.create({
+    //   url: `https://about.downthemall.org/4.0/?cur=${major}`,
+    // });
+  }
+});
+
 locale.then(() => {
-  new class Action extends Handler {
-    constructor() {
-      super();
-      this.onClicked = this.onClicked.bind(this);
-      action.onClicked.addListener(this.onClicked);
-    }
-
-    async onClicked(tab: {id: number}) {
-      if (!tab.id) {
-        return;
-      }
-      try {
-        await this.processResults(
-          true,
-          await runContentJob(
-            tab, "/bundles/content-gather.js", {
-              type: "DTA:gather",
-              selectionOnly: false,
-              textLinks: await Prefs.get("text-links", true),
-              schemes: Array.from(ALLOWED_SCHEMES.values()),
-              transferable: TRANSFERABLE_PROPERTIES,
-            }));
-      }
-      catch (ex) {
-        console.error(ex);
-      }
-    }
-  }();
-
   const menuHandler = new class Menus extends Handler {
     constructor() {
       super();
       this.onClicked = this.onClicked.bind(this);
       const alls = new Map<string, string[]>();
-      const mcreate = (options: any) => {
+      const menuCreate = (options: any) => {
+        if (CHROME) {
+          delete options.icons;
+          options.contexts = options.contexts.
+            filter((e: string) => CHROME_CONTEXTS.has(e));
+          if (!options.contexts.length) {
+            return;
+          }
+        }
         if (options.contexts.includes("all")) {
           alls.set(options.id, options.contexts);
         }
-        return menus.create(options);
+        menus.create(options);
       };
-      mcreate({
+      menuCreate({
         id: "DTARegularLink",
         contexts: ["link"],
         icons: {
@@ -160,7 +194,7 @@ locale.then(() => {
         },
         title: _("dta.regular.link"),
       });
-      mcreate({
+      menuCreate({
         id: "DTATurboLink",
         contexts: ["link"],
         icons: {
@@ -169,7 +203,7 @@ locale.then(() => {
         },
         title: _("dta.turbo.link"),
       });
-      mcreate({
+      menuCreate({
         id: "DTARegularImage",
         contexts: ["image"],
         icons: {
@@ -178,7 +212,7 @@ locale.then(() => {
         },
         title: _("dta.regular.image"),
       });
-      mcreate({
+      menuCreate({
         id: "DTATurboImage",
         contexts: ["image"],
         icons: {
@@ -187,7 +221,7 @@ locale.then(() => {
         },
         title: _("dta.turbo.image"),
       });
-      mcreate({
+      menuCreate({
         id: "DTARegularMedia",
         contexts: ["video", "audio"],
         icons: {
@@ -196,7 +230,7 @@ locale.then(() => {
         },
         title: _("dta.regular.media"),
       });
-      mcreate({
+      menuCreate({
         id: "DTATurboMedia",
         contexts: ["video", "audio"],
         icons: {
@@ -205,7 +239,7 @@ locale.then(() => {
         },
         title: _("dta.turbo.media"),
       });
-      mcreate({
+      menuCreate({
         id: "DTARegularSelection",
         contexts: ["selection"],
         icons: {
@@ -214,7 +248,7 @@ locale.then(() => {
         },
         title: _("dta.regular.selection"),
       });
-      mcreate({
+      menuCreate({
         id: "DTATurboSelection",
         contexts: ["selection"],
         icons: {
@@ -223,7 +257,7 @@ locale.then(() => {
         },
         title: _("dta.turbo.selection"),
       });
-      mcreate({
+      menuCreate({
         id: "DTARegular",
         contexts: ["all", "browser_action", "tools_menu"],
         icons: {
@@ -232,7 +266,7 @@ locale.then(() => {
         },
         title: _("dta.regular"),
       });
-      mcreate({
+      menuCreate({
         id: "DTATurbo",
         contexts: ["all", "browser_action", "tools_menu"],
         icons: {
@@ -241,12 +275,12 @@ locale.then(() => {
         },
         title: _("dta.turbo"),
       });
-      mcreate({
+      menuCreate({
         id: "sep-1",
         contexts: ["all", "browser_action", "tools_menu"],
         type: "separator"
       });
-      mcreate({
+      menuCreate({
         id: "DTARegularAll",
         contexts: ["all", "browser_action", "tools_menu"],
         icons: {
@@ -255,7 +289,7 @@ locale.then(() => {
         },
         title: _("dta-regular-all"),
       });
-      mcreate({
+      menuCreate({
         id: "DTATurboAll",
         contexts: ["all", "browser_action", "tools_menu"],
         icons: {
@@ -267,12 +301,12 @@ locale.then(() => {
       const sep2ctx = menus.ACTION_MENU_TOP_LEVEL_LIMIT === 6 ?
         ["all", "tools_menu"] :
         ["all", "browser_action", "tools_menu"];
-      mcreate({
+      menuCreate({
         id: "sep-2",
         contexts: sep2ctx,
         type: "separator"
       });
-      mcreate({
+      menuCreate({
         id: "DTAAdd",
         contexts: ["all", "browser_action", "tools_menu"],
         icons: {
@@ -283,12 +317,12 @@ locale.then(() => {
         },
         title: _("add-download"),
       });
-      mcreate({
+      menuCreate({
         id: "sep-3",
         contexts: ["all", "browser_action", "tools_menu"],
         type: "separator"
       });
-      mcreate({
+/*      menuCreate({
         id: "DTAManager",
         contexts: ["all", "browser_action", "tools_menu"],
         icons: {
@@ -296,8 +330,8 @@ locale.then(() => {
           32: "/style/button-manager@2x.png",
         },
         title: _("manager.short"),
-      });
-      mcreate({
+      });*/
+      menuCreate({
         id: "DTAPrefs",
         contexts: ["all", "browser_action", "tools_menu"],
         icons: {
@@ -388,7 +422,7 @@ locale.then(() => {
       }
     }
 
-    async enumulate(action: string) {
+    async emulate(action: string) {
       const tab = await tabs.query({
         active: true,
         currentWindow: true,
@@ -510,32 +544,133 @@ locale.then(() => {
     }
   }();
 
-  Bus.on("do-regular", () => menuHandler.enumulate("DTARegular"));
-  Bus.on("do-regular-all", () => menuHandler.enumulate("DTARegularAll"));
-  Bus.on("do-turbo", () => menuHandler.enumulate("DTATurbo"));
-  Bus.on("do-turbo-all", () => menuHandler.enumulate("DTATurboAll"));
+  new class Action extends Handler {
+    constructor() {
+      super();
+      this.onClicked = this.onClicked.bind(this);
+      action.onClicked.addListener(this.onClicked);
+      Prefs.get("button-type", false).then(v => this.adjust(v));
+      Prefs.on("button-type", (prefs, key, value) => {
+        this.adjust(value);
+      });
+    }
+
+    adjust(type: string) {
+      action.setPopup({
+        popup: type !== "popup" ? "" : "/windows/popup.html"
+      });
+      let icons;
+      switch (type) {
+      case "popup":
+        icons = {
+          16: "/style/icon16.png",
+          32: "/style/icon32.png",
+          48: "/style/icon48.png",
+          64: "/style/icon64.png",
+          128: "/style/icon128.png",
+          256: "/style/icon256.png"
+        };
+        break;
+
+      case "dta":
+        icons = {
+          16: "/style/button-regular.png",
+          32: "/style/button-regular@2x.png",
+        };
+        break;
+
+      case "turbo":
+        icons = {
+          16: "/style/button-turbo.png",
+          32: "/style/button-turbo@2x.png",
+        };
+        break;
+
+      case "manager":
+        icons = {
+          16: "/style/button-manager.png",
+          32: "/style/button-manager@2x.png",
+        };
+        break;
+      }
+      action.setIcon({path: icons});
+    }
+
+    async onClicked() {
+      switch (await Prefs.get("button-type")) {
+      case "popup":
+        break;
+
+      case "dta":
+        menuHandler.emulate("DTARegular");
+        break;
+
+      case "turbo":
+        menuHandler.emulate("DTATurbo");
+        break;
+
+      case "manager":
+        menuHandler.emulate("DTAManager");
+        break;
+      }
+    }
+  }();
+
+
+  Bus.on("do-regular", () => menuHandler.emulate("DTARegular"));
+  Bus.on("do-regular-all", () => menuHandler.emulate("DTARegularAll"));
+  Bus.on("do-turbo", () => menuHandler.emulate("DTATurbo"));
+  Bus.on("do-turbo-all", () => menuHandler.emulate("DTATurboAll"));
   Bus.on("do-single", () => API.singleRegular(null));
   Bus.on("open-manager", () => openManager(true));
   Bus.on("open-prefs", () => openPrefs());
 
-  function adjustAction(globalTurbo: boolean) {
-    action.setPopup({
-      popup: globalTurbo ? "" : null
-    });
-    action.setIcon({
-      path: globalTurbo ? {
-        16: "/style/button-turbo.png",
-        32: "/style/button-turbo@2x.png",
-      } : null
+  (async function init() {
+    const urlBase = runtime.getURL("");
+    history.onVisited.addListener(({url}: {url: string}) => {
+      if (!url || !url.startsWith(urlBase)) {
+        return;
+      }
+      history.deleteUrl({url});
     });
+    const results: {url?: string}[] = await history.search({text: urlBase});
+    for (const {url} of results) {
+      if (!url) {
+        continue;
+      }
+      history.deleteUrl({url});
+    }
+
+    if (!CHROME) {
+      const sessionRemover = async () => {
+        for (const s of await sessions.getRecentlyClosed()) {
+          if (s.tab) {
+            if (s.tab.url.startsWith(urlBase)) {
+              await sessions.forgetClosedTab(s.tab.windowId, s.tab.sessionId);
+            }
+            continue;
+          }
+          if (!s.window || !s.window.tabs || s.window.tabs.length > 1) {
+            continue;
+          }
+          const [tab] = s.window.tabs;
+          if (tab.url.startsWith(urlBase)) {
+            await sessions.forgetClosedWindow(s.window.sessionId);
+          }
+        }
+      };
+      sessions.onChanged.addListener(sessionRemover);
+      await sessionRemover();
+    }
+
+    try {
+      await DB.init();
+    }
+    catch (ex) {
+      console.error("db init", ex.toString(), ex.message, ex.stack, ex);
     }
 
-  (async function init() {
     await Prefs.set("last-run", new Date());
-    Prefs.get("global-turbo", false).then(v => adjustAction(v));
-    Prefs.on("global-turbo", (prefs, key, value) => {
-      adjustAction(value);
-    });
     await filters();
     await getManager();
   })().catch(ex => {

lib/batches.ts

@@ -73,7 +73,7 @@ class Numeral implements Generator {
     this.digits = dir ? rawpieces[0].length : rawpieces[1].length;
     this.length = Math.floor(
       (this.stop - this.start + (dir ? 1 : -1)) / this.step);
-    this.preview = this[Symbol.iterator]().next().value;
+    this.preview = this[Symbol.iterator]().next().value as string;
     Object.freeze(this);
   }
 
@@ -93,6 +93,60 @@ class Numeral implements Generator {
   }
 }
 
+class Character implements Generator {
+  public readonly start: number;
+
+  public readonly stop: number;
+
+  public readonly step: number;
+
+  public readonly length: number;
+
+  public readonly preview: string;
+
+  constructor(str: string) {
+    const rawpieces = str.split(":").map(e => e.trim());
+    const pieces = rawpieces.map((e, i) => {
+      if (i === 2) {
+        return reallyParseInt(e);
+      }
+      if (e.length > 1) {
+        throw new Error("Malformed Character sequence");
+      }
+      return e.charCodeAt(0);
+    });
+    if (pieces.length < 2) {
+      throw new Error("Invalid input");
+    }
+    const [start, stop, step] = pieces;
+    if (step === 0) {
+      throw new Error("Invalid step");
+    }
+    this.step = !step ? 1 : step;
+    const dir = this.step > 0;
+    if (dir && start > stop) {
+      throw new Error("Invalid sequence");
+    }
+    else if (!dir && start < stop) {
+      throw new Error("Invalid sequence");
+    }
+    this.start = start;
+    this.stop = stop;
+    this.length = Math.floor(
+      (this.stop - this.start + (dir ? 1 : -1)) / this.step);
+    this.preview = this[Symbol.iterator]().next().value as string;
+    Object.freeze(this);
+  }
+
+  *[Symbol.iterator]() {
+    const {start, stop, step} = this;
+    const dir = step > 0;
+    for (let i = start; (dir ? i <= stop : i >= stop); i += step) {
+      yield String.fromCharCode(i);
+    }
+  }
+}
+
 export class BatchGenerator implements Generator {
   private readonly gens: Generator[];
 
@@ -120,11 +174,16 @@ export class BatchGenerator implements Generator {
       try {
         this.gens.push(new Numeral(tok));
       }
-      catch (ex) {
+      catch {
+        try {
+          this.gens.push(new Character(tok));
+        }
+        catch {
           this.gens.push(new Literal(`[${tok}]`));
           this.hasInvalid = true;
         }
       }
+    }
     if (str) {
       this.gens.push(new Literal(str));
     }

lib/browser.ts

@@ -9,44 +9,112 @@ interface ExtensionListener {
 }
 
 export interface MessageSender {
-  tab?: Tab;
-  frameId?: number;
-  id?: number;
-  url?: string;
-  tlsChannelId?: string;
+  readonly tab?: Tab;
+  readonly frameId?: number;
+  readonly id?: number;
+  readonly url?: string;
+  readonly tlsChannelId?: string;
 }
 
 
 export interface Tab {
-  id?: number;
+  readonly id?: number;
+  readonly incognito?: boolean;
 }
 
 export interface MenuClickInfo {
-  menuItemId: string | number;
-  button?: number;
-  linkUrl?: string;
-  srcUrl?: string;
+  readonly menuItemId: string | number;
+  readonly button?: number;
+  readonly linkUrl?: string;
+  readonly srcUrl?: string;
 }
 
 
 export interface RawPort {
-  error: any;
-  name: string;
-  onDisconnect: ExtensionListener;
-  onMessage: ExtensionListener;
-  sender?: MessageSender;
+  readonly error: any;
+  readonly name: string;
+  readonly sender?: MessageSender;
+  readonly onDisconnect: ExtensionListener;
+  readonly onMessage: ExtensionListener;
   disconnect: () => void;
   postMessage: (message: any) => void;
 }
 
-export const {extension} = polyfill;
-export const {notifications} = polyfill;
+interface WebRequestFilter {
+  urls?: string[];
+}
+
+interface WebRequestListener {
+  addListener(
+    callback: Function,
+    filter: WebRequestFilter,
+    extraInfoSpec: string[]
+    ): void;
+  removeListener(callback: Function): void;
+}
+
+type Header = {name: string; value: string};
+
+export interface DownloadOptions {
+  conflictAction: string;
+  filename?: string;
+  saveAs: boolean;
+  url: string;
+  method?: string;
+  body?: string;
+  incognito?: boolean;
+  headers: Header[];
+}
+
+export interface DownloadsQuery {
+  id?: number;
+}
+
+interface Downloads {
+  download(download: DownloadOptions): Promise<number>;
+  open(manId: number): Promise<void>;
+  show(manId: number): Promise<void>;
+  pause(manId: number): Promise<void>;
+  resume(manId: number): Promise<void>;
+  cancel(manId: number): Promise<void>;
+  erase(query: DownloadsQuery): Promise<void>;
+  search(query: DownloadsQuery): Promise<any[]>;
+  getFileIcon(id: number, options?: any): Promise<string>;
+  setShelfEnabled(state: boolean): void;
+  removeFile(manId: number): Promise<void>;
+  readonly onCreated: ExtensionListener;
+  readonly onChanged: ExtensionListener;
+  readonly onErased: ExtensionListener;
+  readonly onDeterminingFilename?: ExtensionListener;
+}
+
+interface WebRequest {
+  readonly onBeforeSendHeaders: WebRequestListener;
+  readonly onSendHeaders: WebRequestListener;
+  readonly onHeadersReceived: WebRequestListener;
+}
+
+export interface OnInstalled {
+  readonly reason: string;
+  readonly previousVersion?: string;
+  readonly temporary: boolean;
+}
+
 export const {browserAction} = polyfill;
 export const {contextMenus} = polyfill;
-export const {downloads} = polyfill;
+export const {downloads}: {downloads: Downloads} = polyfill;
+export const {extension} = polyfill;
+export const {history} = polyfill;
 export const {menus} = polyfill;
+export const {notifications} = polyfill;
 export const {runtime} = polyfill;
+export const {sessions} = polyfill;
 export const {storage} = polyfill;
 export const {tabs} = polyfill;
 export const {webNavigation} = polyfill;
+export const {webRequest}: {webRequest: WebRequest} = polyfill;
 export const {windows} = polyfill;
+export const {theme} = polyfill;
+
+export const CHROME = navigator.appVersion.includes("Chrome/");
+export const OPERA = navigator.appVersion.includes("OPR/");

lib/bus.ts

@@ -8,32 +8,35 @@ import {runtime, tabs, RawPort, MessageSender} from "./browser";
 export class Port extends EventEmitter {
   private port: RawPort | null;
 
+  private disconnected = false;
+
   constructor(port: RawPort) {
     super();
     this.port = port;
 
-    let disconnected = false;
-    const disconnect = () => {
-      if (disconnected) {
-        return;
-      }
-      disconnected = true;
-      this.port = null; // Break the cycle
-      this.emit("disconnect", this, port);
-    };
     // Nasty firefox bug, thus listen for tab removal explicitly
     if (port.sender && port.sender.tab && port.sender.tab.id) {
       const otherTabId = port.sender.tab.id;
-      const tabListener = function(tabId: number) {
+      const tabListener = (tabId: number) => {
         if (tabId !== otherTabId) {
           return;
         }
-        disconnect();
+        this.disconnect();
       };
       tabs.onRemoved.addListener(tabListener);
     }
     port.onMessage.addListener(this.onMessage.bind(this));
-    port.onDisconnect.addListener(disconnect);
+    port.onDisconnect.addListener(this.disconnect.bind(this));
+  }
+
+  disconnect() {
+    if (this.disconnected) {
+      return;
+    }
+    this.disconnected = true;
+    const {port} = this;
+    this.port = null; // Break the cycle
+    this.emit("disconnect", this, port);
   }
 
   get name() {
@@ -120,6 +123,9 @@ export const Bus = new class extends EventEmitter {
       port.disconnect();
       return;
     }
-    this.ports.emit(port.name, new Port(port));
+    const wrapped = new Port(port);
+    if (!this.ports.emit(port.name, wrapped)) {
+      wrapped.disconnect();
+    }
   }
 }();

lib/cdheaderparser.ts

@@ -0,0 +1,230 @@
+/**
+ * (c) 2017 Rob Wu <rob@robwu.nl> (https://robwu.nl)
+ * This Source Code Form is subject to the terms of the Mozilla Public
+ * License, v. 2.0. If a copy of the MPL was not distributed with this
+ * file, You can obtain one at http://mozilla.org/MPL/2.0/. */
+
+"use strict";
+/* eslint-disable max-len,no-magic-numbers */
+// License: MPL-2
+
+/**
+  * This typescript port was done by Nils Maier based on
+  * https://github.com/Rob--W/open-in-browser/blob/83248155b633ed41bc9cdb1205042653e644abd2/extension/content-disposition.js
+  * Special thanks goes to Rob doing all the heavy lifting and putting
+  * it together in a reuseable, open source'd library.
+  */
+
+const R_RFC6266 = /(?:^|;)\s*filename\*\s*=\s*([^";\s][^;\s]*|"(?:[^"\\]|\\"?)+"?)/i;
+const R_RFC5987 = /(?:^|;)\s*filename\s*=\s*([^";\s][^;\s]*|"(?:[^"\\]|\\"?)+"?)/i;
+
+function unquoteRFC2616(value: string) {
+  if (!value.startsWith("\"")) {
+    return value;
+  }
+
+  const parts = value.slice(1).split("\\\"");
+  // Find the first unescaped " and terminate there.
+  for (let i = 0; i < parts.length; ++i) {
+    const quotindex = parts[i].indexOf("\"");
+    if (quotindex !== -1) {
+      parts[i] = parts[i].slice(0, quotindex);
+      // Truncate and stop the iteration.
+      parts.length = i + 1;
+    }
+    parts[i] = parts[i].replace(/\\(.)/g, "$1");
+  }
+  value = parts.join("\"");
+  return value;
+}
+
+export class CDHeaderParser {
+  private needsFixup: boolean;
+
+  // We need to keep this per instance, because of the global flag.
+  // Hence we need to reset it after a use.
+  private R_MULTI = /(?:^|;)\s*filename\*((?!0\d)\d+)(\*?)\s*=\s*([^";\s][^;\s]*|"(?:[^"\\]|\\"?)+"?)/gi;
+
+  /**
+   * Parse a content-disposition header, with relaxed spec tolerance
+   *
+   * @param {string} header Header to parse
+   * @returns {string} Parsed header
+   */
+  parse(header: string) {
+    this.needsFixup = true;
+
+    // filename*=ext-value ("ext-value" from RFC 5987, referenced by RFC 6266).
+    {
+      const match = R_RFC6266.exec(header);
+      if (match) {
+        const [, tmp] = match;
+        let filename = unquoteRFC2616(tmp);
+        filename = unescape(filename);
+        filename = this.decodeRFC5897(filename);
+        filename = this.decodeRFC2047(filename);
+        return this.maybeFixupEncoding(filename);
+      }
+    }
+
+    // Continuations (RFC 2231 section 3, referenced by RFC 5987 section 3.1).
+    // filename*n*=part
+    // filename*n=part
+    {
+      const tmp = this.getParamRFC2231(header);
+      if (tmp) {
+        // RFC 2047, section
+        const filename = this.decodeRFC2047(tmp);
+        return this.maybeFixupEncoding(filename);
+      }
+    }
+
+    // filename=value (RFC 5987, section 4.1).
+    {
+      const match = R_RFC5987.exec(header);
+      if (match) {
+        const [, tmp] = match;
+        let filename = unquoteRFC2616(tmp);
+        filename = this.decodeRFC2047(filename);
+        return this.maybeFixupEncoding(filename);
+      }
+    }
+    return "";
+  }
+
+  private maybeDecode(encoding: string, value: string) {
+    if (!encoding) {
+      return value;
+    }
+    const bytes = Array.from(value, c => c.charCodeAt(0));
+    if (!bytes.every(code => code <= 0xff)) {
+      return value;
+    }
+    try {
+      value = new TextDecoder(encoding, {fatal: true}).
+        decode(new Uint8Array(bytes));
+      this.needsFixup = false;
+    }
+    catch {
+      // TextDecoder constructor threw - unrecognized encoding.
+    }
+    return value;
+  }
+
+  private maybeFixupEncoding(value: string) {
+    if (!this.needsFixup && /[\x80-\xff]/.test(value)) {
+      return value;
+    }
+
+    // Maybe multi-byte UTF-8.
+    value = this.maybeDecode("utf-8", value);
+    if (!this.needsFixup) {
+      return value;
+    }
+
+    // Try iso-8859-1 encoding.
+    return this.maybeDecode("iso-8859-1", value);
+  }
+
+  private getParamRFC2231(value: string) {
+    const matches: string[][] = [];
+
+    // Iterate over all filename*n= and filename*n*= with n being an integer
+    // of at least zero. Any non-zero number must not start with '0'.
+    let match;
+    this.R_MULTI.lastIndex = 0;
+    while ((match = this.R_MULTI.exec(value)) !== null) {
+      const [, num, quot, part] = match;
+      const n = parseInt(num, 10);
+      if (n in matches) {
+        // Ignore anything after the invalid second filename*0.
+        if (n === 0) {
+          break;
+        }
+        continue;
+      }
+      matches[n] = [quot, part];
+    }
+
+    const parts: string[] = [];
+    for (let n = 0; n < matches.length; ++n) {
+      if (!(n in matches)) {
+        // Numbers must be consecutive. Truncate when there is a hole.
+        break;
+      }
+      const [quot, rawPart] = matches[n];
+      let part = unquoteRFC2616(rawPart);
+      if (quot) {
+        part = unescape(part);
+        if (n === 0) {
+          part = this.decodeRFC5897(part);
+        }
+      }
+      parts.push(part);
+    }
+    return parts.join("");
+  }
+
+  private decodeRFC2047(value: string) {
+    // RFC 2047-decode the result. Firefox tried to drop support for it, but
+    // backed out because some servers use it - https://bugzil.la/875615
+    // Firefox's condition for decoding is here:
+
+    // eslint-disable-next-line max-len
+    // https://searchfox.org/mozilla-central/rev/4a590a5a15e35d88a3b23dd6ac3c471cf85b04a8/netwerk/mime/nsMIMEHeaderParamImpl.cpp#742-748
+
+    // We are more strict and only recognize RFC 2047-encoding if the value
+    // starts with "=?", since then it is likely that the full value is
+    // RFC 2047-encoded.
+
+    // Firefox also decodes words even where RFC 2047 section 5 states:
+    // "An 'encoded-word' MUST NOT appear within a 'quoted-string'."
+
+    // eslint-disable-next-line no-control-regex
+    if (!value.startsWith("=?") || /[\x00-\x19\x80-\xff]/.test(value)) {
+      return value;
+    }
+    // RFC 2047, section 2.4
+    // encoded-word = "=?" charset "?" encoding "?" encoded-text "?="
+    // charset = token (but let's restrict to characters that denote a
+    //           possibly valid encoding).
+    // encoding = q or b
+    // encoded-text = any printable ASCII character other than ? or space.
+    //                ... but Firefox permits ? and space.
+    return value.replace(
+      /=\?([\w-]*)\?([QqBb])\?((?:[^?]|\?(?!=))*)\?=/g,
+      (_, charset, encoding, text) => {
+        if (encoding === "q" || encoding === "Q") {
+          // RFC 2047 section 4.2.
+          text = text.replace(/_/g, " ");
+          text = text.replace(/=([0-9a-fA-F]{2})/g,
+            (_: string, hex: string) => String.fromCharCode(parseInt(hex, 16)));
+          return this.maybeDecode(charset, text);
+        }
+
+        // else encoding is b or B - base64 (RFC 2047 section 4.1)
+        try {
+          text = atob(text);
+        }
+        catch {
+          // ignored
+        }
+        return this.maybeDecode(charset, text);
+      });
+  }
+
+  private decodeRFC5897(extValue: string) {
+    // Decodes "ext-value" from RFC 5987.
+    const extEnd = extValue.indexOf("'");
+    if (extEnd < 0) {
+      // Some servers send "filename*=" without encoding'language' prefix,
+      // e.g. in https://github.com/Rob--W/open-in-browser/issues/26
+      // Let's accept the value like Firefox (57) (Chrome 62 rejects it).
+      return extValue;
+    }
+    const encoding = extValue.slice(0, extEnd);
+    const langvalue = extValue.slice(extEnd + 1);
+    // Ignore language (RFC 5987 section 3.2.1, and RFC 6266 section 4.1 ).
+    return this.maybeDecode(encoding, langvalue.replace(/^[^']*'/, ""));
+  }
+}

lib/db.ts

@@ -1,14 +1,25 @@
 "use strict";
+// License: MIT
 
 // eslint-disable-next-line no-unused-vars
 import { BaseItem } from "./item";
-
-// License: MIT
+// eslint-disable-next-line no-unused-vars
+import { Download } from "./manager/download";
+import { RUNNING, QUEUED, RETRYING } from "./manager/state";
+import { storage } from "./browser";
+import { sort } from "./sorting";
 
 const VERSION = 1;
 const STORE = "queue";
 
-export const DB = new class DB {
+interface Database {
+  init(): Promise<void>;
+  saveItems(items: Download[]): Promise<unknown>;
+  deleteItems(items: any[]): Promise<void>;
+  getAll(): Promise<BaseItem[]>;
+}
+
+export class IDB implements Database {
   private db?: IDBDatabase;
 
   constructor() {
@@ -69,7 +80,7 @@ export const DB = new class DB {
     return await new Promise(this.getAllInternal);
   }
 
-  saveItemsInternal(items: any[], resolve: Function, reject: Function) {
+  saveItemsInternal(items: Download[], resolve: Function, reject: Function) {
     if (!items || !items.length || !this.db) {
       resolve();
       return;
@@ -83,9 +94,13 @@ export const DB = new class DB {
         if (item.private) {
           continue;
         }
-        const req = store.put(item.toJSON());
+        const json = item.toJSON();
+        if (item.state === RUNNING || item.state === RETRYING) {
+          json.state = QUEUED;
+        }
+        const req = store.put(json);
         if (!("dbId" in item) || item.dbId < 0) {
-          req.onsuccess = () => item.dbId = req.result;
+          req.onsuccess = () => item.dbId = req.result as number;
         }
       }
     }
@@ -94,12 +109,12 @@ export const DB = new class DB {
     }
   }
 
-  async saveItems(items: any[]) {
+  async saveItems(items: Download[]) {
     await this.init();
     return await new Promise(this.saveItemsInternal.bind(this, items));
   }
 
-  deleteItemsInternal(items: any[], resolve: Function, reject: Function) {
+  deleteItemsInternal(items: any[], resolve: () => void, reject: Function) {
     if (!items || !items.length || !this.db) {
       resolve();
       return;
@@ -132,4 +147,120 @@ export const DB = new class DB {
     await this.init();
     await new Promise(this.deleteItemsInternal.bind(this, items));
   }
+}
+
+class StorageDB implements Database {
+  private counter = 1;
+
+  async init(): Promise<void> {
+    const {db = null} = await storage.local.get("db");
+    if (!db || !db.counter) {
+      return;
+    }
+    this.counter = db.counter;
+  }
+
+  async saveItems(items: Download[]) {
+    const db: any = {items: []};
+    for (const item of items) {
+      if (!item.dbId) {
+        item.dbId = ++this.counter;
+      }
+      db.items.push(item.toJSON());
+    }
+    db.counter = this.counter;
+    await storage.local.set({db});
+  }
+
+  async deleteItems(items: any[]): Promise<void> {
+    const gone = new Set(items.map(i => i.dbId));
+    const {db = null} = await storage.local.get("db");
+    if (!db) {
+      return;
+    }
+    db.items = db.items.filter((i: any) => !gone.has(i.dbId));
+    await storage.local.set({db});
+  }
+
+  async getAll() {
+    const {db = null} = await storage.local.get("db");
+    if (!db || !Array.isArray(db.items)) {
+      return [];
+    }
+    return sort(db.items, (i: any) => i.position) as BaseItem[];
+  }
+}
+
+class MemoryDB implements Database {
+  private counter = 1;
+
+  private items = new Map();
+
+  init(): Promise<void> {
+    return Promise.resolve();
+  }
+
+  saveItems(items: Download[]) {
+    for (const item of items) {
+      if (item.private) {
+        continue;
+      }
+      if (!item.dbId) {
+        item.dbId = ++this.counter;
+      }
+      this.items.set(item.dbId, item.toJSON());
+    }
+    return Promise.resolve();
+  }
+
+  deleteItems(items: any[]) {
+    for (const item of items) {
+      if (!("dbId" in item)) {
+        continue;
+      }
+      this.items.delete(item.dbId);
+    }
+    return Promise.resolve();
+  }
+
+  getAll(): Promise<BaseItem[]> {
+    return Promise.resolve(Array.from(this.items.values()));
+  }
+}
+
+export const DB = new class DBWrapper implements Database {
+  saveItems(items: Download[]): Promise<unknown> {
+    return this.db.saveItems(items);
+  }
+
+  deleteItems(items: any[]): Promise<void> {
+    return this.db.deleteItems(items);
+  }
+
+  getAll(): Promise<BaseItem[]> {
+    return this.db.getAll();
+  }
+
+  private db: Database;
+
+  async init() {
+    try {
+      this.db = new IDB();
+      await this.db.init();
+    }
+    catch (ex) {
+      console.warn(
+        "Failed to initialize idb backend, using storage db fallback", ex);
+      try {
+        this.db = new StorageDB();
+        await this.db.init();
+      }
+      catch (ex) {
+        console.warn(
+          "Failed to initialize storage backend, using memory db fallback", ex);
+        this.db = new MemoryDB();
+        await this.db.init();
+      }
+    }
+  }
 }();

lib/filters.ts

@@ -9,7 +9,7 @@ import { EventEmitter } from "./events";
 import { TYPE_LINK, TYPE_MEDIA, TYPE_ALL } from "./constants";
 // eslint-disable-next-line no-unused-vars
 import { Overlayable } from "./objectoverlay";
-import * as DEFAULT_FILTERS from "../data/filters.json";
+import DEFAULT_FILTERS from "../data/filters.json";
 import { FASTFILTER } from "./recentlist";
 import { _, locale } from "./i18n";
 // eslint-disable-next-line no-unused-vars
@@ -94,7 +94,7 @@ function *parseIntoRegexpInternal(str: string): Iterable<RegExp> {
 
   // multi-expression
   if (str.includes(",")) {
-    for (const part in str.split(",")) {
+    for (const part of str.split(",")) {
       yield *parseIntoRegexpInternal(part);
     }
     return;

lib/i18n.ts

@@ -2,6 +2,19 @@
 // License: MIT
 
 import {memoize} from "./memoize";
+import langs from "../_locales/all.json";
+import { sorted, naturalCaseCompare } from "./sorting";
+import lf from "localforage";
+
+export const ALL_LANGS = Object.freeze(new Map<string, string>(
+  sorted(Object.entries(langs), e => {
+    return [e[1], e[0]];
+  }, naturalCaseCompare)));
+
+let CURRENT = "en";
+export function getCurrentLanguage() {
+  return CURRENT;
+}
 
 declare let browser: any;
 declare let chrome: any;
@@ -9,6 +22,8 @@ declare let chrome: any;
 const CACHE_KEY = "_cached_locales";
 const CUSTOM_KEY = "_custom_locale";
 
+const normalizer = /[^A-Za-z0-9_]/g;
+
 interface JSONEntry {
   message: string;
   placeholders: any;
@@ -25,11 +40,11 @@ class Entry {
     this.message = entry.message.replace(/\$[A-Z0-9]+\$/g, (r: string) => {
       hit = true;
       const id = r.substr(1, r.length - 2).toLocaleLowerCase();
-      const pholder = entry.placeholders[id];
-      if (!pholder || !pholder.content) {
+      const placeholder = entry.placeholders[id];
+      if (!placeholder || !placeholder.content) {
         throw new Error(`Invalid placeholder: ${id}`);
       }
-      return `${pholder.content}$`;
+      return `${placeholder.content}$`;
     });
     if (!hit) {
       throw new Error("Not entry-able");
@@ -72,7 +87,7 @@ class Localization {
   }
 
   localize(id: string, ...args: any[]) {
-    const entry = this.strings.get(id);
+    const entry = this.strings.get(id.replace(normalizer, "_"));
     if (!entry) {
       return "";
     }
@@ -108,31 +123,50 @@ async function fetchLanguage(code: string) {
 }
 
 
-function loadCached() {
-  if (document.location.pathname.includes("/windows/")) {
-    const cached = localStorage.getItem(CACHE_KEY);
-    if (cached) {
-      return JSON.parse(cached) as any[];
-    }
-  }
+async function loadCached(): Promise<any> {
+  const cached = await lf.getItem<string>(CACHE_KEY);
+  if (!cached) {
     return null;
+  }
+  const parsed = JSON.parse(cached);
+  if (!Array.isArray(parsed) || !parsed[0].CRASH || !parsed[0].CRASH.message) {
+    console.warn("rejecting cached locales", parsed);
+    return null;
+  }
+  return parsed;
 }
 
 async function loadRawLocales() {
-  // en is the base locale
+  // en is the base locale, always to be loaded
+  // The loader will override string from it with more specific string
+  // from other locales
   const langs = new Set<string>(["en"]);
-  const ui = (browser.i18n || chrome.i18n).getUILanguage();
-  langs.add(ui);
 
-  // Try the base too
-  if (ui.includes("-")) {
-    langs.add(ui.split(/[-]+/)[0]);
-  }
-  else if (ui.includes("_")) {
-    langs.add(ui.split(/[_]+/)[0]);
+  const uiLang: string = (typeof browser !== "undefined" ? browser : chrome).
+    i18n.getUILanguage();
+
+  // Chrome will only look for underscore versions of locale codes,
+  // while Firefox will look for both.
+  // So we better normalize the code to the underscore version.
+  // However, the API seems to always return the dash-version.
+
+  // Add all base locales into ascending order of priority,
+  // starting with the most unspecific base locale, ending
+  // with the most specific locale.
+  // e.g. this will transform ["zh", "CN"] -> ["zh", "zh_CN"]
+  uiLang.split(/[_-]/g).reduce<string[]>((prev, curr) => {
+    prev.push(curr);
+    langs.add(prev.join("_"));
+    return prev;
+  }, []);
+
+  if (CURRENT && CURRENT !== "default") {
+    langs.delete(CURRENT);
+    langs.add(CURRENT);
   }
 
-  const fetched = await Promise.all(Array.from(langs, fetchLanguage));
+  const valid = Array.from(langs).filter(e => ALL_LANGS.has(e));
+  const fetched = await Promise.all(Array.from(valid, fetchLanguage));
   return fetched.filter(e => !!e);
 }
 
@@ -140,17 +174,32 @@ async function load(): Promise<Localization> {
   try {
     checkBrowser();
     try {
+      let currentLang: any = "";
+      if (typeof browser !== "undefined") {
+        currentLang = await browser.storage.sync.get("language");
+      }
+      else {
+        currentLang = await new Promise(
+          resolve => chrome.storage.sync.get("language", resolve));
+      }
+      if ("language" in currentLang) {
+        currentLang = currentLang.language;
+      }
+      if (!currentLang || !currentLang.length) {
+        currentLang = "default";
+      }
+      CURRENT = currentLang;
       // en is the base locale
-      let valid = loadCached();
+      let valid = await loadCached();
       if (!valid) {
         valid = await loadRawLocales();
-        localStorage.setItem(CACHE_KEY, JSON.stringify(valid));
+        await lf.setItem(CACHE_KEY, JSON.stringify(valid));
       }
       if (!valid.length) {
-        throw new Error("Could not lood ANY of these locales");
+        throw new Error("Could not load ANY of these locales");
       }
 
-      const custom = localStorage.getItem(CUSTOM_KEY);
+      const custom = await lf.getItem<string>(CUSTOM_KEY);
       if (custom) {
         try {
           valid.push(JSON.parse(custom));
@@ -193,7 +242,7 @@ locale.then(l => {
 /**
  * Localize a message
  * @param {string} id Identifier of the string to localize
- * @param {string[]} [subst] Message substituations
+ * @param {string[]} [subst] Message substitutions
  * @returns {string} Localized message
  */
 export function _(id: string, ...subst: any[]) {
@@ -256,11 +305,11 @@ export async function localize<T extends HTMLElement | DocumentFragment>(
   return localize_(elem);
 }
 
-export function saveCustomLocale(data?: string) {
+export async function saveCustomLocale(data?: string) {
   if (!data) {
-    localStorage.removeItem(CUSTOM_KEY);
+    await lf.removeItem(CUSTOM_KEY);
     return;
   }
   new Localization(JSON.parse(data));
-  localStorage.setItem(CUSTOM_KEY, data);
+  await localStorage.setItem(CUSTOM_KEY, data);
 }

lib/iconcache.ts

@@ -0,0 +1,124 @@
+"use strict";
+// License: MIT
+
+import { downloads, CHROME } from "./browser";
+import { EventEmitter } from "../uikit/lib/events";
+import { PromiseSerializer } from "./pserializer";
+import lf from "localforage";
+
+
+const STORE = "iconcache";
+
+// eslint-disable-next-line no-magic-numbers
+const CACHE_SIZES = CHROME ? [16, 32] : [16, 32, 64, 127];
+
+const BLACKLISTED = Object.freeze(new Set([
+  "",
+  "ext",
+  "ico",
+  "pif",
+  "scr",
+  "ani",
+  "cur",
+  "ttf",
+  "otf",
+  "woff",
+  "woff2",
+  "cpl",
+  "desktop",
+  "app",
+]));
+
+async function getIcon(size: number, manId: number) {
+  const raw = await downloads.getFileIcon(manId, {size});
+  const icon = new URL(raw);
+  if (icon.protocol === "data:") {
+    const res = await fetch(icon.toString());
+    const blob = await res.blob();
+    return {size, icon: blob};
+  }
+  return {size, icon};
+}
+
+const SYNONYMS = Object.freeze(new Map<string, string>([
+  ["jpe", "jpg"],
+  ["jpeg", "jpg"],
+  ["jfif", "jpg"],
+  ["mpe", "mpg"],
+  ["mpeg", "mpg"],
+  ["m4v", "mp4"],
+]));
+
+export const IconCache = new class IconCache extends EventEmitter {
+  private db = lf.createInstance({name: STORE});
+
+  private cache: Map<string, string>;
+
+  constructor() {
+    super();
+    this.cache = new Map();
+    this.get = PromiseSerializer.wrapNew(8, this, this.get);
+    this.set = PromiseSerializer.wrapNew(1, this, this.set);
+  }
+
+  private normalize(ext: string) {
+    ext = ext.toLocaleLowerCase();
+    return SYNONYMS.get(ext) || ext;
+  }
+
+  // eslint-disable-next-line no-magic-numbers
+  async get(ext: string, size = 16) {
+    ext = this.normalize(ext);
+    if (BLACKLISTED.has(ext)) {
+      return undefined;
+    }
+    const sext = `${ext}-${size}`;
+    let rv = this.cache.get(sext);
+    if (rv) {
+      return rv;
+    }
+    rv = this.cache.get(sext);
+    if (rv) {
+      return rv;
+    }
+    let result = await this.db.getItem<any>(sext);
+    if (!result) {
+      return this.cache.get(sext);
+    }
+    rv = this.cache.get(sext);
+    if (rv) {
+      return rv;
+    }
+    if (typeof result !== "string") {
+      result = URL.createObjectURL(result).toString();
+    }
+
+    this.cache.set(sext, result);
+    this.cache.set(ext, "");
+    return result;
+  }
+
+  async set(ext: string, manId: number) {
+    ext = this.normalize(ext);
+    if (BLACKLISTED.has(ext)) {
+      return;
+    }
+    if (this.cache.has(ext)) {
+      // already processed in this session
+      return;
+    }
+    // eslint-disable-next-line no-magic-numbers
+    const urls = await Promise.all(CACHE_SIZES.map(
+      size => getIcon(size, manId)));
+    if (this.cache.has(ext)) {
+      // already processed in this session
+      return;
+    }
+    for (const {size, icon} of urls) {
+      this.cache.set(`${ext}-${size}`, URL.createObjectURL(icon));
+      await this.db.setItem(`${ext}-${size}`, icon);
+    }
+    this.cache.set(ext, "");
+    this.emit("cached", ext);
+  }
+}();

lib/imex.ts

@@ -0,0 +1,261 @@
+"use strict";
+// License: MIT
+
+import { getTextLinks } from "./textlinks";
+// eslint-disable-next-line no-unused-vars
+import { BaseItem } from "./item";
+import { ALLOWED_SCHEMES } from "./constants";
+
+export const NS_METALINK_RFC5854 = "urn:ietf:params:xml:ns:metalink";
+export const NS_DTA = "http://www.downthemall.net/properties#";
+
+function parseNum(
+    file: Element,
+    attr: string,
+    defaultValue: number,
+    ns = NS_METALINK_RFC5854) {
+  const val = file.getAttributeNS(ns, attr);
+  if (!val) {
+    return defaultValue + 1;
+  }
+  const num = parseInt(val, 10);
+  if (isFinite(num)) {
+    return num;
+  }
+  return defaultValue + 1;
+}
+
+function importMeta4(data: string) {
+  const parser = new DOMParser();
+  const document = parser.parseFromString(data, "text/xml");
+  const {documentElement} = document;
+  const items: BaseItem[] = [];
+  let batch = 0;
+  for (const file of documentElement.querySelectorAll("file")) {
+    try {
+      const url = Array.from(file.querySelectorAll("url")).map(u => {
+        try {
+          const {textContent} = u;
+          if (!textContent) {
+            return null;
+          }
+          const url = new URL(textContent);
+          if (!ALLOWED_SCHEMES.has(url.protocol)) {
+            return null;
+          }
+          const prio = parseNum(u, "priority", 0);
+          return {
+            url,
+            prio
+          };
+        }
+        catch {
+          return null;
+        }
+      }).filter(u => !!u).reduce((p, c) => {
+        if (!c) {
+          return null;
+        }
+        if (!p || p.prio < c.prio) {
+          return c;
+        }
+        return p;
+      });
+      if (!url) {
+        continue;
+      }
+      batch = parseNum(file, "num", batch, NS_DTA);
+      const idx = parseNum(file, "idx", 0, NS_DTA);
+      const item: BaseItem = {
+        url: url.url.toString(),
+        usable: decodeURIComponent(url.url.toString()),
+        batch,
+        idx
+      };
+      const ref = file.getAttributeNS(NS_DTA, "referrer");
+      if (ref) {
+        item.referrer = ref;
+        item.usableReferrer = decodeURIComponent(ref);
+      }
+      const mask = file.getAttributeNS(NS_DTA, "mask");
+      if (mask) {
+        item.mask = mask;
+      }
+      const description = file.querySelector("description");
+      if (description && description.textContent) {
+        item.description = description.textContent.trim();
+      }
+      const title = file.getElementsByTagNameNS(NS_DTA, "title");
+      if (title && title[0] && title[0].textContent) {
+        item.title = title[0].textContent;
+      }
+      items.push(item);
+    }
+    catch (ex) {
+      console.error("Failed to import file", ex);
+    }
+  }
+  return items;
+}
+
+function parseKV(current: BaseItem, line: string) {
+  const [k, v] = line.split("=", 2);
+  switch (k.toLocaleLowerCase().trim()) {
+  case "referer": {
+    const refererUrls = getTextLinks(v);
+    if (refererUrls && refererUrls.length) {
+      current.referrer = refererUrls.pop();
+      current.usableReferrer = decodeURIComponent(current.referrer || "");
+    }
+    break;
+  }
+  }
+}
+
+export function importText(data: string) {
+  if (data.includes(NS_METALINK_RFC5854)) {
+    return importMeta4(data);
+  }
+  const splitter = /((?:.|\r)+)\n|(.+)$/g;
+  const spacer = /^\s+/;
+  let match;
+  let current: BaseItem | undefined = undefined;
+  let idx = 0;
+  const items = [];
+  while ((match = splitter.exec(data)) !== null) {
+    try {
+      const line = match[0].trimRight();
+      if (!line) {
+        continue;
+      }
+      if (spacer.test(line)) {
+        if (!current) {
+          continue;
+        }
+        parseKV(current, line);
+        continue;
+      }
+      const urls = getTextLinks(line);
+      if (!urls || !urls.length) {
+        continue;
+      }
+      current = {
+        url: urls[0],
+        usable: decodeURIComponent(urls[0]),
+        idx: ++idx
+      };
+      items.push(current);
+    }
+    catch (ex) {
+      current = undefined;
+      console.error("Failed to import", ex);
+    }
+  }
+  return items;
+}
+
+export interface Exporter {
+  fileName: string;
+  getText(items: BaseItem[]): string;
+}
+
+class TextExporter {
+  readonly fileName: string;
+
+  constructor() {
+    this.fileName = "links.txt";
+  }
+
+  getText(items: BaseItem[]) {
+    const lines = [];
+    for (const item of items) {
+      lines.push(item.url);
+    }
+    return lines.join("\n");
+  }
+}
+
+class Aria2Exporter {
+  readonly fileName: string;
+
+  constructor() {
+    this.fileName = "links.aria2.txt";
+  }
+
+  getText(items: BaseItem[]) {
+    const lines = [];
+    for (const item of items) {
+      lines.push(item.url);
+      if (item.referrer) {
+        lines.push(`  referer=${item.referrer}`);
+      }
+    }
+    return lines.join("\n");
+  }
+}
+
+class MetalinkExporter {
+  readonly fileName: string;
+
+  constructor() {
+    this.fileName = "links.meta4";
+  }
+
+  getText(items: BaseItem[]) {
+    const document = window.document.implementation.
+      createDocument(NS_METALINK_RFC5854, "metalink", null);
+    const root = document.documentElement;
+    root.setAttributeNS(NS_DTA, "generator", "TraitorousDownloading!");
+    root.appendChild(document.createComment(
+      "metalink as exported by TraitorousDownloading!",
+    ));
+
+    for (const item of items) {
+      const anyItem = item as any;
+      const f = document.createElementNS(NS_METALINK_RFC5854, "file");
+      f.setAttribute("name", anyItem.currentName);
+      if (item.batch) {
+        f.setAttributeNS(NS_DTA, "num", item.batch.toString());
+      }
+      if (item.idx) {
+        f.setAttributeNS(NS_DTA, "idx", item.idx.toString());
+      }
+      if (item.referrer) {
+        f.setAttributeNS(NS_DTA, "referrer", item.referrer);
+      }
+      if (item.mask) {
+        f.setAttributeNS(NS_DTA, "mask", item.mask);
+      }
+
+      if (item.description) {
+        const n = document.createElementNS(NS_METALINK_RFC5854, "description");
+        n.textContent = item.description;
+        f.appendChild(n);
+      }
+
+      if (item.title) {
+        const n = document.createElementNS(NS_DTA, "title");
+        n.textContent = item.title;
+        f.appendChild(n);
+      }
+
+      const u = document.createElementNS(NS_METALINK_RFC5854, "url");
+      u.textContent = item.url;
+      f.appendChild(u);
+
+      if (anyItem.totalSize > 0) {
+        const s = document.createElementNS(NS_METALINK_RFC5854, "size");
+        s.textContent = anyItem.totalSize.toString();
+        f.appendChild(s);
+      }
+      root.appendChild(f);
+    }
+    let xml = "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n";
+    xml += root.outerHTML;
+    return xml;
+  }
+}
+
+export const textExporter = new TextExporter();
+export const aria2Exporter = new Aria2Exporter();
+export const metalinkExporter = new MetalinkExporter();

lib/ipreg.ts

@@ -0,0 +1,4 @@
+"use strict";
+// License: MIT
+
+export const IPReg = /^(?:(?:[0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])[.]){3}(?:[0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])$|^(?:(?:(?:[0-9a-fA-F]{1,4}):){7}(?:(?:[0-9a-fA-F]{1,4})|:)|(?:(?:[0-9a-fA-F]{1,4}):){6}(?:((?:[0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])[.]){3}(?:[0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])|:(?:[0-9a-fA-F]{1,4})|:)|(?:(?:[0-9a-fA-F]{1,4}):){5}(?::((?:[0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])[.]){3}(?:[0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])|(:(?:[0-9a-fA-F]{1,4})){1,2}|:)|(?:(?:[0-9a-fA-F]{1,4}):){4}(?:(:(?:[0-9a-fA-F]{1,4})){0,1}:((?:[0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])[.]){3}(?:[0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])|(:(?:[0-9a-fA-F]{1,4})){1,3}|:)|(?:(?:[0-9a-fA-F]{1,4}):){3}(?:(:(?:[0-9a-fA-F]{1,4})){0,2}:((?:[0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])[.]){3}(?:[0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])|(:(?:[0-9a-fA-F]{1,4})){1,4}|:)|(?:(?:[0-9a-fA-F]{1,4}):){2}(?:(:(?:[0-9a-fA-F]{1,4})){0,3}:((?:[0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])[.]){3}(?:[0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])|(:(?:[0-9a-fA-F]{1,4})){1,5}|:)|(?:(?:[0-9a-fA-F]{1,4}):){1}(?:(:(?:[0-9a-fA-F]{1,4})){0,4}:((?:[0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])[.]){3}(?:[0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])|(:(?:[0-9a-fA-F]{1,4})){1,6}|:)|(?::((?::(?:[0-9a-fA-F]{1,4})){0,5}:((?:[0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])[.]){3}(?:[0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])|(?::(?:[0-9a-fA-F]{1,4})){1,7}|:)))(%[0-9a-zA-Z]{1,})?$/;

lib/item.ts

@@ -15,6 +15,7 @@ export interface BaseItem {
   batch?: number;
   idx: number;
   mask?: string;
+  subfolder?: string;
   startDate?: number;
   private?: boolean;
   postData?: string;
@@ -27,10 +28,12 @@ const OPTIONPROPS = Object.freeze([
   "fileName",
   "batch", "idx",
   "mask",
+  "subfolder",
   "startDate",
   "private",
   "postData",
-  "paused"
+  "paused",
+  "server", "cookies",
 ]);
 
 function maybeAssign(options: any, what: any) {

lib/manager/basedownload.ts

@@ -5,6 +5,8 @@
 import { parsePath, URLd } from "../util";
 import { QUEUED, RUNNING, PAUSED } from "./state";
 import Renamer from "./renamer";
+// eslint-disable-next-line no-unused-vars
+import { BaseItem } from "../item";
 
 const SAVEDPROPS = [
   "state",
@@ -14,6 +16,7 @@ const SAVEDPROPS = [
   "usableReferrer",
   "fileName",
   "mask",
+  "subfolder",
   "date",
   // batches
   "batch",
@@ -27,6 +30,9 @@ const SAVEDPROPS = [
   "written",
   // server stuff
   "serverName",
+  "browserName",
+  "mime",
+  "prerolled",
   // other options
   "private",
   // db
@@ -39,10 +45,15 @@ const DEFAULTS = {
   state: QUEUED,
   error: "",
   serverName: "",
+  browserName: "",
   fileName: "",
   totalSize: 0,
   written: 0,
   manId: 0,
+  mime: "",
+  prerolled: false,
+  retries: 0,
+  deadline: 0
 };
 
 let sessionId = 0;
@@ -59,14 +70,26 @@ export class BaseDownload {
 
   public url: string;
 
+  public usable: string;
+
   public uReferrer: URLd;
 
   public referrer: string;
 
+  public usableReferrer: string;
+
   public startDate: Date;
 
   public fileName: string;
 
+  public description?: string;
+
+  public title?: string;
+
+  public batch: number;
+
+  public idx: number;
+
   public error: string;
 
   public postData: any;
@@ -79,10 +102,19 @@ export class BaseDownload {
 
   public serverName: string;
 
+  public browserName: string;
+
+  public mime: string;
+
   public mask: string;
 
+  public subfolder: string;
 
-  constructor(options: any) {
+  public prerolled: boolean;
+
+  public retries: number;
+
+  constructor(options: BaseItem) {
     Object.assign(this, DEFAULTS);
     this.assign(options);
     if (this.state === RUNNING) {
@@ -90,14 +122,16 @@ export class BaseDownload {
     }
     this.sessionId = ++sessionId;
     this.renamer = new Renamer(this);
+    this.retries = 0;
   }
 
-  assign(options: any) {
+  assign(options: BaseItem) {
     // eslint-disable-next-line @typescript-eslint/no-this-alias
     const self: any = this;
+    const other: any = options;
     for (const prop of SAVEDPROPS) {
       if (prop in options) {
-        self[prop] = options[prop];
+        self[prop] = other[prop];
       }
     }
     this.uURL = new URL(this.url) as URLd;
@@ -115,6 +149,10 @@ export class BaseDownload {
     return this.serverName || this.fileName || this.urlName || "index.html";
   }
 
+  get currentName() {
+    return this.browserName || this.dest.name || this.finalName;
+  }
+
   get urlName() {
     const path = parsePath(this.uURL);
     if (path.name) {
@@ -152,7 +190,11 @@ export class BaseDownload {
     rv.destName = dest.name;
     rv.destPath = dest.path;
     rv.destFull = dest.full;
+    rv.currentName = this.browserName || rv.destName || rv.finalName;
+    rv.currentFull = `${dest.path}/${rv.currentName}`;
     rv.error = this.error;
+    rv.ext = this.renamer.p_ext;
+    rv.retries = this.retries;
     return rv;
   }
 }

lib/manager/download.ts

@@ -1,35 +1,43 @@
 "use strict";
 // License: MIT
 
-import { Prefs } from "../prefs";
-import { parsePath, filterInSitu } from "../util";
-import {
-  QUEUED, RUNNING, CANCELED, PAUSED, MISSING, DONE,
-  FORCABLE, PAUSABLE, CANCELABLE,
-} from "./state";
-import { BaseDownload } from "./basedownload";
+// eslint-disable-next-line no-unused-vars
+import { CHROME, downloads, DownloadOptions } from "../browser";
+import { Prefs, PrefWatcher } from "../prefs";
 import { PromiseSerializer } from "../pserializer";
+import { filterInSitu, parsePath } from "../util";
+import { BaseDownload } from "./basedownload";
 // eslint-disable-next-line no-unused-vars
 import { Manager } from "./man";
-import { downloads } from "../browser";
+import Renamer from "./renamer";
+import {
+  CANCELABLE,
+  CANCELED,
+  DONE,
+  FORCABLE,
+  MISSING,
+  PAUSEABLE,
+  PAUSED,
+  QUEUED,
+  RUNNING,
+  RETRYING
+} from "./state";
+// eslint-disable-next-line no-unused-vars
+import { Preroller, PrerollResults } from "./preroller";
 
+function isRecoverable(error: string) {
+  switch (error) {
+  case "SERVER_FAILED":
+    return true;
 
-const setShelfEnabled = downloads.setShelfEnabled || function() {
-  // ignored
-};
-
-type Header = {name: string; value: string};
-interface Options {
-  conflictAction: string;
-  filename: string;
-  saveAs: boolean;
-  url: string;
-  method?: string;
-  body?: string;
-  incognito: boolean;
-  headers: Header[];
+  default:
+    return error.startsWith("NETWORK_");
+  }
 }
 
+const RETRIES = new PrefWatcher("retries", 5);
+const RETRY_TIME = new PrefWatcher("retry-time", 5);
+
 export class Download extends BaseDownload {
   public manager: Manager;
 
@@ -41,6 +49,10 @@ export class Download extends BaseDownload {
 
   public error: string;
 
+  public dbId: number;
+
+  public deadline: number;
+
   constructor(manager: Manager, options: any) {
     super(options);
     this.manager = manager;
@@ -50,6 +62,7 @@ export class Download extends BaseDownload {
   }
 
   markDirty() {
+    this.renamer = new Renamer(this);
     this.manager.setDirty(this);
   }
 
@@ -71,22 +84,28 @@ export class Download extends BaseDownload {
     if (this.manId) {
       const {manId: id} = this;
       try {
-        const state = await downloads.search({id});
-        if (state[0].state === "in_progress") {
+        const state = (await downloads.search({id})).pop() || {};
+        if (state.state === "in_progress" && !state.error && !state.paused) {
           this.changeState(RUNNING);
           this.updateStateFromBrowser();
           return;
         }
-        if (!state[0].canResume) {
+        if (state.state === "complete") {
+          this.changeState(DONE);
+          this.updateStateFromBrowser();
+          return;
+        }
+        if (!state.canResume) {
           throw new Error("Cannot resume");
         }
         // Cannot await here
         // Firefox bug: will not return until download is finished
-        downloads.resume(id).catch(() => {});
+        downloads.resume(id).catch(console.error);
         this.changeState(RUNNING);
         return;
       }
       catch (ex) {
+        console.error("cannot resume", ex);
         this.manager.removeManId(this.manId);
         this.removeFromBrowser();
       }
@@ -94,33 +113,54 @@ export class Download extends BaseDownload {
     if (this.state !== QUEUED) {
       throw new Error("invalid state");
     }
-    console.trace("starting", this.toString(), this.toMsg());
+    console.log("starting", this.toString(), this.toMsg());
     this.changeState(RUNNING);
+
+    // Do NOT await
+    this.reallyStart();
+  }
+
+  private async reallyStart() {
     try {
-      const options: Options = {
+      if (!this.prerolled) {
+        await this.maybePreroll();
+        if (this.state !== RUNNING) {
+          // Aborted by preroll
+          return;
+        }
+      }
+      const options: DownloadOptions = {
         conflictAction: await Prefs.get("conflict-action"),
-        filename: this.dest.full,
         saveAs: false,
         url: this.url,
         headers: [],
-        incognito: this.private
       };
+      if (!CHROME) {
+        options.filename = this.dest.full;
+      }
+      if (!CHROME && this.private) {
+        options.incognito = true;
+      }
       if (this.postData) {
         options.body = this.postData;
         options.method = "POST";
       }
-      if (this.referrer) {
+      if (!CHROME && this.referrer) {
         options.headers.push({
           name: "Referer",
           value: this.referrer
         });
       }
+      else if (CHROME) {
+        options.headers.push({
+          name: "X-DTA-ID",
+          value: this.sessionId.toString(),
+        });
+      }
       if (this.manId) {
         this.manager.removeManId(this.manId);
       }
 
-      setShelfEnabled(false);
-      try {
       try {
         this.manager.addManId(
           this.manId = await downloads.download(options), this);
@@ -134,10 +174,6 @@ export class Download extends BaseDownload {
         this.manager.addManId(
           this.manId = await downloads.download(options), this);
       }
-      }
-      finally {
-        setShelfEnabled(true);
-      }
       this.markDirty();
     }
     catch (ex) {
@@ -147,6 +183,45 @@ export class Download extends BaseDownload {
     }
   }
 
+  private async maybePreroll() {
+    try {
+      if (this.prerolled) {
+        // Check again, just in case, async and all
+        return;
+      }
+      const roller = new Preroller(this);
+      if (!roller.shouldPreroll) {
+        return;
+      }
+      const res = await roller.roll();
+      if (!res) {
+        return;
+      }
+      this.adoptPrerollResults(res);
+    }
+    catch (ex) {
+      console.error("Failed to preroll", this, ex.toString(), ex.stack, ex);
+    }
+    finally {
+      if (this.state === RUNNING) {
+        this.prerolled = true;
+        this.markDirty();
+      }
+    }
+  }
+
+  adoptPrerollResults(res: PrerollResults) {
+    if (res.mime) {
+      this.mime = res.mime;
+    }
+    if (res.name) {
+      this.serverName = res.name;
+    }
+    if (res.error) {
+      this.cancelAccordingToError(res.error);
+    }
+  }
+
   resume(forced = false) {
     if (!(FORCABLE & this.state)) {
       return;
@@ -159,26 +234,41 @@ export class Download extends BaseDownload {
     }
   }
 
-  async pause() {
-    if (!(PAUSABLE & this.state)) {
+  async pause(retry?: boolean) {
+    if (!(PAUSEABLE & this.state)) {
       return;
     }
+
+    if (!retry) {
+      this.retries = 0;
+      this.deadline = 0;
+    }
+    else {
+      // eslint-disable-next-line no-magic-numbers
+      this.deadline = Date.now() + RETRY_TIME.value * 60 * 1000;
+    }
+
     if (this.state === RUNNING && this.manId) {
       try {
         await downloads.pause(this.manId);
       }
       catch (ex) {
         console.error("pause", ex.toString(), ex);
+        this.cancel();
         return;
       }
     }
-    this.changeState(PAUSED);
+
+    this.changeState(retry ? RETRYING : PAUSED);
   }
 
   reset() {
+    this.prerolled = false;
     this.manId = 0;
     this.written = this.totalSize = 0;
-    this.serverName = "";
+    this.mime = this.serverName = this.browserName = "";
+    this.retries = 0;
+    this.deadline = 0;
   }
 
   async removeFromBrowser() {
@@ -187,7 +277,7 @@ export class Download extends BaseDownload {
       await downloads.cancel(id);
     }
     catch (ex) {
-      // ingored
+      // ignored
     }
     await new Promise(r => setTimeout(r, 1000));
     try {
@@ -195,7 +285,7 @@ export class Download extends BaseDownload {
     }
     catch (ex) {
       console.error(id, ex.toString(), ex);
-      // ingored
+      // ignored
     }
   }
 
@@ -211,6 +301,17 @@ export class Download extends BaseDownload {
     this.changeState(CANCELED);
   }
 
+  async cancelAccordingToError(error: string) {
+    if (!isRecoverable(error) || ++this.retries > RETRIES.value) {
+      this.cancel();
+      this.error = error;
+      return;
+    }
+
+    await this.pause(true);
+    this.error = error;
+  }
+
   setMissing() {
     if (this.manId) {
       this.manager.removeManId(this.manId);
@@ -255,14 +356,19 @@ export class Download extends BaseDownload {
       const state = (await downloads.search({id: this.manId})).pop();
       const {filename, error} = state;
       const path = parsePath(filename);
-      this.serverName = path.name;
+      this.browserName = path.name;
       this.adoptSize(state);
+      if (!this.mime && state.mime) {
+        this.mime = state.mime;
+      }
       this.markDirty();
       switch (state.state) {
       case "in_progress":
-        if (error) {
-          this.cancel();
-          this.error = error;
+        if (state.paused) {
+          this.changeState(PAUSED);
+        }
+        else if (error) {
+          this.cancelAccordingToError(error);
         }
         else {
           this.changeState(RUNNING);
@@ -273,6 +379,9 @@ export class Download extends BaseDownload {
         if (state.paused) {
           this.changeState(PAUSED);
         }
+        else if (error) {
+          this.cancelAccordingToError(error);
+        }
         else {
           this.cancel();
           this.error = error || "";
@@ -289,4 +398,27 @@ export class Download extends BaseDownload {
       this.setMissing();
     }
   }
+
+  updateFromSuggestion(state: any) {
+    const res: PrerollResults = {};
+    if (state.mime) {
+      res.mime = state.mime;
+    }
+    if (state.filename) {
+      res.name = state.filename;
+    }
+    if (state.finalUrl) {
+      res.finalURL = state.finalUrl;
+      const detected = Preroller.maybeFindNameFromSearchParams(this, res);
+      if (detected) {
+        res.name = detected;
+      }
+    }
+    try {
+      this.adoptPrerollResults(res);
+    }
+    finally {
+      this.markDirty();
+    }
+  }
 }

lib/manager/man.ts

@@ -4,30 +4,40 @@
 import { EventEmitter } from "../events";
 import { Notification } from "../notifications";
 import { DB } from "../db";
-import { QUEUED, CANCELED, RUNNING } from "./state";
+import { QUEUED, CANCELED, RUNNING, RETRYING } from "./state";
 // eslint-disable-next-line no-unused-vars
 import { Bus, Port } from "../bus";
 import { sort } from "../sorting";
-import { Prefs } from "../prefs";
+import { Prefs, PrefWatcher } from "../prefs";
 import { _ } from "../i18n";
 import { CoalescedUpdate, mapFilterInSitu, filterInSitu } from "../util";
 import { PromiseSerializer } from "../pserializer";
-import {Download} from "./download";
-import {ManagerPort} from "./port";
-import {Scheduler} from "./scheduler";
-import {Limits} from "./limits";
-import { downloads } from "../browser";
+import { Download } from "./download";
+import { ManagerPort } from "./port";
+import { Scheduler } from "./scheduler";
+import { Limits } from "./limits";
+import { downloads, runtime, webRequest, CHROME, OPERA } from "../browser";
+import { browser } from "webextension-polyfill-ts";
 
+const US = runtime.getURL("");
 
 const AUTOSAVE_TIMEOUT = 2000;
 const DIRTY_TIMEOUT = 100;
 // eslint-disable-next-line no-magic-numbers
 const MISSING_TIMEOUT = 12 * 1000;
+const RELOAD_TIMEOUT = 10 * 1000;
+
+const setShelfEnabled = downloads.setShelfEnabled || function() {
+  // ignored
+};
+
+const FINISH_NOTIFICATION = new PrefWatcher("finish-notification", true);
+const SOUNDS = new PrefWatcher("sounds", false);
 
 export class Manager extends EventEmitter {
   private items: Download[];
 
-  private active: boolean;
+  public active: boolean;
 
   private notifiedFinished: boolean;
 
@@ -43,38 +53,63 @@ export class Manager extends EventEmitter {
 
   private readonly running: Set<Download>;
 
+  private readonly retrying: Set<Download>;
+
   private scheduler: Scheduler | null;
 
+  private shouldReload: boolean;
+
+  private deadlineTimer: number;
+
   constructor() {
+    if (!document.location.href.includes("background")) {
+      throw new Error("Not on background");
+    }
     super();
     this.active = true;
+    this.shouldReload = false;
     this.notifiedFinished = true;
     this.items = [];
     this.saveQueue = new CoalescedUpdate(
       AUTOSAVE_TIMEOUT, this.save.bind(this));
     this.dirty = new CoalescedUpdate(
       DIRTY_TIMEOUT, this.processDirty.bind(this));
+    this.processDeadlines = this.processDeadlines.bind(this);
     this.sids = new Map();
     this.manIds = new Map();
     this.ports = new Set();
     this.scheduler = null;
     this.running = new Set();
+    this.retrying = new Set();
 
     this.startNext = PromiseSerializer.wrapNew(1, this, this.startNext);
 
     downloads.onChanged.addListener(this.onChanged.bind(this));
     downloads.onErased.addListener(this.onErased.bind(this));
+    if (CHROME && downloads.onDeterminingFilename) {
+      downloads.onDeterminingFilename.addListener(
+        this.onDeterminingFilename.bind(this));
+    }
 
     Bus.onPort("manager", (port: Port) => {
-      const mport = new ManagerPort(this, port);
+      const managerPort = new ManagerPort(this, port);
       port.on("disconnect", () => {
-        this.ports.delete(mport);
+        this.ports.delete(managerPort);
       });
-      this.ports.add(mport);
+      this.ports.add(managerPort);
+      return true;
     });
     Limits.on("changed", () => {
       this.resetScheduler();
     });
+
+    if (CHROME) {
+      webRequest.onBeforeSendHeaders.addListener(
+        this.stuffReferrer.bind(this),
+        {urls: ["<all_urls>"]},
+        ["blocking", "requestHeaders", "extraHeaders"]
+      );
+    }
   }
 
   async init() {
@@ -88,9 +123,19 @@ export class Manager extends EventEmitter {
       }
       this.items.push(rv);
     });
-    await this.resetScheduler();
-    this.emit("inited");
+
+    // Do not wait for the scheduler
+    this.resetScheduler();
+
+    this.emit("initialized");
     setTimeout(() => this.checkMissing(), MISSING_TIMEOUT);
+    runtime.onUpdateAvailable.addListener(() => {
+      if (this.running.size) {
+        this.shouldReload = true;
+        return;
+      }
+      runtime.reload();
+    });
     return this;
   }
 
@@ -121,6 +166,20 @@ export class Manager extends EventEmitter {
     this.manIds.delete(downloadId);
   }
 
+  onDeterminingFilename(state: any, suggest: Function) {
+    const download = this.manIds.get(state.id);
+    if (!download) {
+      return;
+    }
+    try {
+      download.updateFromSuggestion(state);
+    }
+    finally {
+      const suggestion = {filename: download.dest.full};
+      suggest(suggestion);
+    }
+  }
+
   async resetScheduler() {
     this.scheduler = null;
     await this.startNext();
@@ -136,7 +195,7 @@ export class Manager extends EventEmitter {
       }
       const next = await this.scheduler.next(this.running);
       if (!next) {
-        this.maybeNotifyFinished();
+        this.maybeRunFinishActions();
         break;
       }
       if (this.running.has(next) || next.state !== QUEUED) {
@@ -156,19 +215,43 @@ export class Manager extends EventEmitter {
   async startDownload(download: Download) {
     // Add to running first, so we don't confuse the scheduler and other parts
     this.running.add(download);
+    setShelfEnabled(false);
     await download.start();
     this.notifiedFinished = false;
   }
 
-  async maybeNotifyFinished() {
-    if (!(await Prefs.get("finish-notification"))) {
+  maybeRunFinishActions() {
+    if (this.running.size) {
       return;
     }
-    if (this.notifiedFinished || this.running.size) {
+    this.maybeNotifyFinished();
+    if (this.shouldReload) {
+      this.saveQueue.trigger();
+      setTimeout(() => {
+        if (this.running.size) {
           return;
         }
+        runtime.reload();
+      }, RELOAD_TIMEOUT);
+    }
+    setShelfEnabled(true);
+  }
+
+  maybeNotifyFinished() {
+    if (this.notifiedFinished || this.running.size || this.retrying.size) {
+      return;
+    }
+    if (SOUNDS.value && !OPERA) {
+      const audio = new Audio(runtime.getURL("/style/done.opus"));
+      audio.addEventListener("canplaythrough", () => audio.play());
+      audio.addEventListener("ended", () => document.body.removeChild(audio));
+      audio.addEventListener("error", () => document.body.removeChild(audio));
+      document.body.appendChild(audio);
+    }
+    if (FINISH_NOTIFICATION.value) {
+      new Notification(null, _("queue-finished"));
+    }
     this.notifiedFinished = true;
-    new Notification(null, _("queue-finished"));
   }
 
   addManId(id: number, download: Download) {
@@ -179,6 +262,61 @@ export class Manager extends EventEmitter {
     this.manIds.delete(id);
   }
 
+  async prepareItems(items: any[]) {
+    var links = new Array();
+    for (var item of items) {
+      var cookiesToSend = Array();
+      var cs = await browser.cookies.getAll({
+        url: item.url,
+        firstPartyDomain: null,
+      });
+
+      if (item.cookies) {
+        for (var c of cs){
+            cookiesToSend.push({
+                name: c.name,
+                value: c.value,
+                domain: c.domain,
+                // expires: new Date(c.expirationDate * 1000) ?? null,
+                path: c.path,
+                secure: c.secure,
+                httponly: c.httpOnly,
+                // samesite: c.sameSite,
+            });
+        }
+      }
+
+      var status = item.paused ? "Paused":"Queue";
+
+      links.push({
+        description: item.description,
+        filename: item.fileName,
+        mask: item.mask,
+        status: status,
+        postData: item.postData,
+        subdir: item.subfolder,
+        title: item.title,
+        url: item.usable,
+        referrer: item.usableReferrer,
+        cookies: cookiesToSend,
+      });
+    }
+    return links;
+  }
+
+  addNewDownloads(items: any[]) {
+    if (!items || !items.length) {
+      return;
+    }
+    this.prepareItems(items).then(links => {
+      var xhr = new XMLHttpRequest();
+      xhr.open("POST", items[0].server);
+      xhr.setRequestHeader("Content-Type", "application/javascript");
+      xhr.send(JSON.stringify(links));
+    });
+  }
+
+/*
   addNewDownloads(items: any[]) {
     if (!items || !items.length) {
       return;
@@ -201,6 +339,7 @@ export class Manager extends EventEmitter {
     this.save(items);
     this.startNext();
   }
+*/
 
   setDirty(item: Download) {
     this.dirty.add(item);
@@ -216,7 +355,7 @@ export class Manager extends EventEmitter {
     this.emit("dirty", items);
   }
 
-  save(items: Download[]) {
+  private save(items: Download[]) {
     DB.saveItems(items.filter(i => !i.removed)).
       catch(console.error);
   }
@@ -267,35 +406,86 @@ export class Manager extends EventEmitter {
     if (oldState === RUNNING) {
       this.running.delete(download);
     }
+    else if (oldState === RETRYING) {
+      this.retrying.delete(download);
+      this.findDeadline();
+    }
     if (newState === QUEUED) {
       this.resetScheduler();
       this.startNext().catch(console.error);
     }
     else if (newState === RUNNING) {
-      // Usually we already added it. Buit if a user uses the built-in
-      // download manager to resart
+      // Usually we already added it. But if a user uses the built-in
+      // download manager to restart
       // a download, we have not, so make sure it is added either way
       this.running.add(download);
     }
     else {
+      if (newState === RETRYING) {
+        this.addRetry(download);
+      }
       this.startNext().catch(console.error);
     }
   }
 
+  addRetry(download: Download) {
+    this.retrying.add(download);
+    this.findDeadline();
+  }
+
+  private findDeadline() {
+    let deadline = Array.from(this.retrying).
+      reduce<number>((deadline, item) => {
+        if (deadline) {
+          return item.deadline ? Math.min(deadline, item.deadline) : deadline;
+        }
+        return item.deadline;
+      }, 0);
+    if (deadline <= 0) {
+      return;
+    }
+    deadline -= Date.now();
+    if (deadline <= 0) {
+      return;
+    }
+
+    if (this.deadlineTimer) {
+      window.clearTimeout(this.deadlineTimer);
+    }
+    this.deadlineTimer = window.setTimeout(this.processDeadlines, deadline);
+  }
+
+  private processDeadlines() {
+    this.deadlineTimer = 0;
+    try {
+      const now = Date.now();
+      this.items.forEach(item => {
+        if (item.deadline && Math.abs(item.deadline - now) < 1000) {
+          this.retrying.delete(item);
+          item.resume(false);
+        }
+      });
+    }
+    finally {
+      this.findDeadline();
+    }
+  }
+
   sorted(sids: number[]) {
     try {
       // Construct new items
-      const csids = new Map(this.sids);
+      const currentSids = new Map(this.sids);
       let items = mapFilterInSitu(sids, sid => {
-        const item = csids.get(sid);
+        const item = currentSids.get(sid);
         if (!item) {
           return null;
         }
-        csids.delete(sid);
+        currentSids.delete(sid);
         return item;
       }, e => !!e);
-      if (csids.size) {
-        items = items.concat(sort(Array.from(csids.values()), i => i.position));
+      if (currentSids.size) {
+        items = items.concat(
+          sort(Array.from(currentSids.values()), i => i.position));
       }
       this.items = items;
       this.setPositions();
@@ -341,6 +531,35 @@ export class Manager extends EventEmitter {
     }
     this.emit("active", this.active);
   }
+
+  getMsgItems() {
+    return this.items.map(e => e.toMsg());
+  }
+
+  stuffReferrer(details: any): any {
+    if (details.tabId > 0 && !US.startsWith(details.initiator)) {
+      return undefined;
+    }
+    const sidx = details.requestHeaders.findIndex(
+      (e: any) => e.name.toLowerCase() === "x-dta-id");
+    if (sidx < 0) {
+      return undefined;
+    }
+    const sid = parseInt(details.requestHeaders[sidx].value, 10);
+    details.requestHeaders.splice(sidx, 1);
+    const item = this.sids.get(sid);
+    if (!item) {
+      return undefined;
+    }
+    details.requestHeaders.push({
+      name: "Referer",
+      value: (item.uReferrer || item.uURL).toString()
+    });
+    const rv: any = {
+      requestHeaders: details.requestHeaders
+    };
+    return rv;
+  }
 }
 
 let inited: Promise<Manager>;

lib/manager/port.ts

@@ -5,6 +5,12 @@ import { donate, openPrefs } from "../windowutils";
 import { API } from "../api";
 // eslint-disable-next-line no-unused-vars
 import { BaseDownload } from "./basedownload";
+// eslint-disable-next-line no-unused-vars
+import { Manager } from "./man";
+// eslint-disable-next-line no-unused-vars
+import { Port } from "../bus";
+// eslint-disable-next-line no-unused-vars
+import { BaseItem } from "../item";
 
 type SID = {sid: number};
 type SIDS = {
@@ -13,9 +19,9 @@ type SIDS = {
 };
 
 export class ManagerPort {
-  private manager: any;
+  private manager: Manager;
 
-  private port: any;
+  private port: Port;
 
   constructor(manager: any, port: any) {
     this.manager = manager;
@@ -38,6 +44,9 @@ export class ManagerPort {
     port.on("prefs", () => {
       openPrefs();
     });
+    port.on("import", ({items}: {items: BaseItem[]}) => {
+      API.regular(items, []);
+    });
     port.on("all", () => this.sendAll());
     port.on("removeSids", this.onMsgRemoveSids);
     port.on("showSingle", async () => {
@@ -61,6 +70,7 @@ export class ManagerPort {
       delete this.manager;
       delete this.port;
     });
+
     this.port.post("active", this.manager.active);
     this.sendAll();
   }
@@ -78,7 +88,6 @@ export class ManagerPort {
   }
 
   sendAll() {
-    this.port.post(
-      "all", this.manager.items.map((e: BaseDownload) => e.toMsg()));
+    this.port.post("all", this.manager.getMsgItems());
   }
 }

lib/manager/preroller.ts

@@ -0,0 +1,252 @@
+"use strict";
+// License: MIT
+
+import MimeType from "whatwg-mimetype";
+// eslint-disable-next-line no-unused-vars
+import { Download } from "./download";
+import { CHROME, webRequest } from "../browser";
+import { CDHeaderParser } from "../cdheaderparser";
+import { sanitizePath, parsePath } from "../util";
+import { MimeDB } from "../mime";
+
+const PREROLL_HEURISTICS = /dl|attach|download|name|file|get|retr|^n$|\.(php|asp|py|pl|action|htm|shtm)/i;
+const PREROLL_HOSTS = /4cdn|chan/;
+const PREROLL_TIMEOUT = 10000;
+const PREROLL_NOPE = new Set<string>();
+
+/* eslint-disable no-magic-numbers */
+const NOPE_STATUSES = Object.freeze(new Set([
+  400,
+  401,
+  402,
+  405,
+  416,
+]));
+/* eslint-enable no-magic-numbers */
+
+const PREROLL_SEARCHEXTS = Object.freeze(new Set<string>([
+  "php",
+  "asp",
+  "aspx",
+  "inc",
+  "py",
+  "pl",
+  "action",
+  "htm",
+  "html",
+  "shtml"
+]));
+const NAME_TESTER = /\.[a-z0-9]{1,5}$/i;
+const CDPARSER = new CDHeaderParser();
+
+export interface PrerollResults {
+  error?: string;
+  name?: string;
+  mime?: string;
+  finalURL?: string;
+}
+
+export class Preroller {
+  private readonly download: Download
+
+  constructor(download: Download) {
+    this.download = download;
+  }
+
+  get shouldPreroll() {
+    if (CHROME) {
+      return false;
+    }
+    const {uURL, renamer} = this.download;
+    const {pathname, search, host} = uURL;
+    if (PREROLL_NOPE.has(host)) {
+      return false;
+    }
+    if (!renamer.p_ext) {
+      return true;
+    }
+    if (search.length) {
+      return true;
+    }
+    if (uURL.pathname.endsWith("/")) {
+      return true;
+    }
+    if (PREROLL_HEURISTICS.test(pathname)) {
+      return true;
+    }
+    if (PREROLL_HOSTS.test(host)) {
+      return true;
+    }
+    return false;
+  }
+
+  async roll() {
+    try {
+      return await (CHROME ? this.prerollChrome() : this.prerollFirefox());
+    }
+    catch (ex) {
+      console.error("Failed to preroll", this, ex.toString(), ex.stack, ex);
+    }
+    return null;
+  }
+
+  private async prerollFirefox() {
+    const controller = new AbortController();
+    const {signal} = controller;
+    const {uURL, uReferrer} = this.download;
+    const res = await fetch(uURL.toString(), {
+      method: "GET",
+      headers: new Headers({
+        Range: "bytes=0-1",
+      }),
+      mode: "same-origin",
+      signal,
+      referrer: (uReferrer || uURL).toString(),
+    });
+    if (res.body) {
+      res.body.cancel();
+    }
+    controller.abort();
+    const {headers} = res;
+    return this.finalize(headers, res);
+  }
+
+  private async prerollChrome() {
+    let rid = "";
+    const {uURL, uReferrer} = this.download;
+    const rurl = uURL.toString();
+    let listener: any;
+    const wr = new Promise<any[]>(resolve => {
+      listener = (details: any) => {
+        const {url, requestId, statusCode} = details;
+        if (rid !== requestId && url !== rurl) {
+          return;
+        }
+        // eslint-disable-next-line no-magic-numbers
+        if (statusCode >= 300 && statusCode < 400) {
+          // Redirect, continue tracking;
+          rid = requestId;
+          return;
+        }
+        resolve(details.responseHeaders);
+      };
+      webRequest.onHeadersReceived.addListener(
+        listener, {urls: ["<all_urls>"]}, ["responseHeaders"]);
+    });
+    const p = Promise.race([
+      wr,
+      new Promise<any[]>((_, reject) =>
+        setTimeout(() => reject(new Error("timeout")), PREROLL_TIMEOUT))
+    ]);
+
+    p.finally(() => {
+      webRequest.onHeadersReceived.removeListener(listener);
+    });
+    const controller = new AbortController();
+    const {signal} = controller;
+    const res = await fetch(rurl, {
+      method: "GET",
+      headers: new Headers({
+        "Range": "bytes=0-1",
+        "X-DTA-ID": this.download.sessionId.toString(),
+      }),
+      signal,
+      referrer: (uReferrer || uURL).toString(),
+    });
+    if (res.body) {
+      res.body.cancel();
+    }
+    controller.abort();
+    const headers = await p;
+    return this.finalize(
+      new Headers(headers.map(i => [i.name, i.value])), res);
+  }
+
+  private finalize(headers: Headers, res: Response): PrerollResults {
+    const rv: PrerollResults = {};
+
+    const type = MimeType.parse(headers.get("content-type") || "");
+    if (type) {
+      rv.mime = type.essence;
+    }
+
+    const dispHeader = headers.get("content-disposition");
+    if (dispHeader) {
+      const file = CDPARSER.parse(dispHeader);
+      // Sanitize
+      rv.name = sanitizePath(file.replace(/[/\\]+/g, "-"));
+    }
+    else {
+      const detected = Preroller.maybeFindNameFromSearchParams(
+        this.download, rv);
+      if (detected) {
+        rv.name = detected;
+      }
+    }
+
+    rv.finalURL = res.url;
+
+    /* eslint-disable no-magic-numbers */
+    const {status} = res;
+    if (status === 404) {
+      rv.error = "SERVER_BAD_CONTENT";
+    }
+    else if (status === 403) {
+      rv.error = "SERVER_FORBIDDEN";
+    }
+    else if (status === 402 || status === 407) {
+      rv.error = "SERVER_UNAUTHORIZED";
+    }
+    else if (NOPE_STATUSES.has(status)) {
+      PREROLL_NOPE.add(this.download.uURL.host);
+      if (PREROLL_NOPE.size > 1000) {
+        PREROLL_NOPE.delete(PREROLL_NOPE.keys().next().value);
+      }
+    }
+    else if (status > 400 && status < 500) {
+      rv.error = "SERVER_FAILED";
+    }
+    /* eslint-enable no-magic-numbers */
+
+    return rv;
+  }
+
+
+  static maybeFindNameFromSearchParams(
+      download: Download, res: PrerollResults) {
+    const {p_ext: ext} = download.renamer;
+    if (ext && !PREROLL_SEARCHEXTS.has(ext.toLocaleLowerCase())) {
+      return undefined;
+    }
+    return Preroller.findNameFromSearchParams(download.uURL, res.mime);
+  }
+
+  static findNameFromSearchParams(url: URL, mimetype?: string) {
+    const {searchParams} = url;
+    let detected = "";
+    for (const [, value] of searchParams) {
+      if (!NAME_TESTER.test(value)) {
+        continue;
+      }
+      const p = parsePath(value);
+      if (!p.base || !p.ext) {
+        continue;
+      }
+      if (!MimeDB.hasExtension(p.ext)) {
+        continue;
+      }
+      if (mimetype) {
+        const mime = MimeDB.getMime(mimetype);
+        if (mime && !mime.extensions.has(p.ext.toLowerCase())) {
+          continue;
+        }
+      }
+      const sanitized = sanitizePath(p.name);
+      if (sanitized.length <= detected.length) {
+        continue;
+      }
+      detected = sanitized;
+    }
+    return detected;
+  }
+}

lib/manager/renamer.ts

@@ -2,8 +2,12 @@
 "use strict";
 // License: MIT
 
-import { parsePath, sanitizePath } from "../util";
 import { _ } from "../i18n";
+import { MimeDB } from "../mime";
+// eslint-disable-next-line no-unused-vars
+import { parsePath, PathInfo, sanitizePath } from "../util";
+// eslint-disable-next-line no-unused-vars
+import { BaseDownload } from "./basedownload";
 
 const REPLACE_EXPR = /\*\w+\*/gi;
 
@@ -22,21 +26,41 @@ const DATE_FORMATTER = new Intl.NumberFormat(undefined, {
 });
 
 export default class Renamer {
-  private readonly d: any;
+  private readonly d: BaseDownload;
 
-  constructor(download: any) {
+  private readonly nameinfo: PathInfo;
+
+  constructor(download: BaseDownload) {
     this.d = download;
+    const info = parsePath(this.d.finalName);
+    this.nameinfo = this.fixupExtension(info);
   }
 
-  get nameinfo() {
-    return parsePath(this.d.finalName);
+  private fixupExtension(info: PathInfo): PathInfo {
+    if (!this.d.mime) {
+      return info;
+    }
+    const mime = MimeDB.getMime(this.d.mime);
+    if (!mime) {
+      return info;
+    }
+    const {ext} = info;
+    if (mime.major === "image" || mime.major === "video") {
+      if (ext && mime.extensions.has(ext.toLowerCase())) {
+        return info;
+      }
+      return new PathInfo(info.base, mime.primary, info.path);
+    }
+    if (ext) {
+      return info;
+    }
+    return new PathInfo(info.base, mime.primary, info.path);
   }
 
   get ref() {
     return this.d.uReferrer;
   }
 
-
   get p_name() {
     return this.nameinfo.base;
   }
@@ -169,24 +193,24 @@ export default class Renamer {
   }
 
   toString() {
-    const {mask} = this.d;
+    const {mask, subfolder} = this.d;
     // eslint-disable-next-line @typescript-eslint/no-this-alias
     const self: any = this;
-    // XXX flat
-    return sanitizePath(mask.replace(REPLACE_EXPR, function(type: string) {
+    const baseMask = subfolder ? `${subfolder}/${mask}` : mask;
+    return sanitizePath(baseMask.replace(REPLACE_EXPR, function(type: string) {
       let prop = type.substr(1, type.length - 2);
       const flat = prop.startsWith("flat");
       if (flat) {
         prop = prop.substr(4);
       }
       prop = `p_${prop}`;
-      const rv = (prop in self) ?
+      let rv = (prop in self) ?
         (self[prop] || "").trim() :
         type;
       if (flat) {
-        return rv.replace(/\/+/g, "-");
+        rv = rv.replace(/[/\\]+/g, "-");
       }
-      return rv;
+      return rv.replace(/\/{2,}/g, "/");
     }));
   }
 }

lib/manager/state.ts

@@ -8,8 +8,9 @@ export const PAUSED = 1 << 3;
 export const DONE = 1 << 4;
 export const CANCELED = 1 << 5;
 export const MISSING = 1 << 6;
+export const RETRYING = 1 << 7;
 
-export const RESUMABLE = PAUSED | CANCELED;
-export const FORCABLE = PAUSED | QUEUED | CANCELED;
-export const PAUSABLE = QUEUED | CANCELED | RUNNING;
-export const CANCELABLE = QUEUED | RUNNING | PAUSED | DONE | MISSING;
+export const RESUMABLE = PAUSED | CANCELED | RETRYING;
+export const FORCABLE = PAUSED | QUEUED | CANCELED | RETRYING;
+export const PAUSEABLE = QUEUED | CANCELED | RUNNING | RETRYING;
+export const CANCELABLE = QUEUED | RUNNING | PAUSED | DONE | MISSING | RETRYING;

lib/mime.ts

@@ -0,0 +1,65 @@
+"use strict";
+// License: MIT
+
+import mime from "../data/mime.json";
+
+export class MimeInfo {
+  public readonly type: string;
+
+  public readonly extensions: Set<string>;
+
+  public readonly major: string;
+
+  public readonly minor: string;
+
+  public readonly primary: string;
+
+  constructor(type: string, extensions: string[]) {
+    this.type = type;
+    const [major, minor] = type.split("/", 2);
+    this.major = major;
+    this.minor = minor;
+    [this.primary] = extensions;
+    this.extensions = new Set(extensions);
+    Object.freeze(this);
+  }
+}
+
+export const MimeDB = new class MimeDB {
+  private readonly mimeToExts: Map<string, MimeInfo>;
+
+  private readonly registeredExtensions: Set<string>;
+
+  constructor() {
+    const exts = new Map<string, string[]>();
+    for (const [prim, more] of Object.entries(mime.e)) {
+      let toadd = more;
+      if (!Array.isArray(toadd)) {
+        toadd = [toadd];
+      }
+      toadd.unshift(prim);
+      exts.set(prim, toadd);
+    }
+    this.mimeToExts = new Map(Array.from(
+      Object.entries(mime.m),
+      ([mime, prim]) => [mime, new MimeInfo(mime, exts.get(prim) || [prim])]
+    ));
+    const all = Array.from(
+      this.mimeToExts.values(),
+      m => Array.from(m.extensions, e => e.toLowerCase()));
+    this.registeredExtensions = new Set(all.flat());
+  }
+
+  getPrimary(mime: string) {
+    const info = this.mimeToExts.get(mime.trim().toLocaleLowerCase());
+    return info ? info.primary : "";
+  }
+
+  getMime(mime: string) {
+    return this.mimeToExts.get(mime.trim().toLocaleLowerCase());
+  }
+
+  hasExtension(ext: string) {
+    return this.registeredExtensions.has(ext.toLowerCase());
+  }
+}();

lib/notifications.ts

@@ -8,7 +8,7 @@ import {EventEmitter} from "./events";
 const DEFAULTS = {
   type: "basic",
   iconUrl: extension.getURL("/style/icon64.png"),
-  title: "DownThemAll!",
+  title: "TraitorousDownloading!",
   message: "message",
 };
 
@@ -25,7 +25,7 @@ export class Notification extends EventEmitter {
     super();
 
     this.generated = !id;
-    id = id || `DownThemAll-notification${++gid}`;
+    id = id || `TraitorousDownloading-notification${++gid}`;
     if (typeof options === "string") {
       options = {message: options};
     }

lib/prefs.ts

@@ -1,9 +1,9 @@
 "use strict";
 // License: MIT
 
-import * as DEFAULT_PREFS from "../data/prefs.json";
+import DEFAULT_PREFS from "../data/prefs.json";
 import { EventEmitter } from "./events";
-import {loadOverlay} from "./objectoverlay";
+import { loadOverlay } from "./objectoverlay";
 import { storage } from "./browser";
 
 const PREFS = Symbol("PREFS");
@@ -99,6 +99,5 @@ export class PrefWatcher {
 
   changed(prefs: any, key: string, value: any) {
     this.value = value;
-    return true;
   }
 }

lib/recentlist.ts

@@ -116,3 +116,14 @@ export const FASTFILTER = new RecentList("fastfilter", [
   "*.z??, *.css, *.html"
 ]);
 FASTFILTER.init().catch(console.error);
+
+export const SUBFOLDER = new RecentList("subfolder", [
+  "",
+  "downthemall",
+]);
+SUBFOLDER.init().catch(console.error);
+
+export const SERVER = new RecentList("server", [
+  "",
+]);
+SERVER.init().catch(console.error);

lib/select.ts

@@ -9,11 +9,12 @@ import { donate, openPrefs, openUrls } from "./windowutils";
 // eslint-disable-next-line no-unused-vars
 import { filters, FAST, Filter } from "./filters";
 import { WindowStateTracker } from "./windowstatetracker";
-import { windows } from "./browser";
+import { windows, CHROME } from "./browser";
 // eslint-disable-next-line no-unused-vars
 import { BaseItem } from "./item";
 
 interface BaseMatchedItem extends BaseItem {
+  sidx?: number;
   matched?: string | null;
   prevMatched?: string | null;
 }
@@ -28,7 +29,8 @@ function computeSelection(
     items: BaseMatchedItem[],
     onlyFast: boolean): ItemDelta[] {
   let ws = items.map((item, idx: number) => {
-    item.idx = idx;
+    item.idx = item.idx || idx;
+    item.sidx = item.sidx || idx;
     const {matched = null} = item;
     item.prevMatched = matched;
     item.matched = null;
@@ -52,7 +54,7 @@ function computeSelection(
   }
   return items.filter(item => item.prevMatched !== item.matched).map(item => {
     return {
-      idx: item.idx,
+      idx: item.sidx as number,
       matched: item.matched
     };
   });
@@ -98,9 +100,16 @@ export async function select(links: BaseItem[], media: BaseItem[]) {
     type: "popup",
   });
   const window = await windows.create(windowOptions);
+  tracker.track(window.id);
   try {
+    if (!CHROME) {
+      windows.update(window.id, tracker.getOptions({}));
+    }
     const port = await Promise.race<Port>([
-      new Promise<Port>(resolve => Bus.oncePort("select", resolve)),
+      new Promise<Port>(resolve => Bus.oncePort("select", port => {
+        resolve(port);
+        return true;
+      })),
       timeout<Port>(5 * 1000)]);
     if (!port.isSelf) {
       throw Error("Invalid sender connected");
@@ -186,8 +195,8 @@ export async function select(links: BaseItem[], media: BaseItem[]) {
       openPrefs();
     });
 
-    port.on("openUrls", ({urls}) => {
-      openUrls(urls);
+    port.on("openUrls", ({urls, incognito}) => {
+      openUrls(urls, incognito);
     });
 
     try {

lib/single.ts

@@ -6,7 +6,7 @@ import { Bus, Port } from "./bus";
 import { WindowStateTracker } from "./windowstatetracker";
 import { Promised, timeout } from "./util";
 import { donate } from "./windowutils";
-import { windows } from "./browser";
+import { windows, CHROME } from "./browser";
 // eslint-disable-next-line no-unused-vars
 import { BaseItem } from "./item";
 
@@ -21,9 +21,16 @@ export async function single(item: BaseItem | null) {
     type: "popup",
   });
   const window = await windows.create(windowOptions);
+  tracker.track(window.id);
   try {
+    if (!CHROME) {
+      windows.update(window.id, tracker.getOptions({}));
+    }
     const port: Port = await Promise.race<Port>([
-      new Promise<Port>(resolve => Bus.oncePort("single", resolve)),
+      new Promise<Port>(resolve => Bus.oncePort("single", port => {
+        resolve(port);
+        return true;
+      })),
       timeout<Port>(5 * 1000)]);
     if (!port.isSelf) {
       throw Error("Invalid sender connected");

lib/util.ts

@@ -2,8 +2,9 @@
 // License: MIT
 
 import * as psl from "psl";
-import {memoize, identity} from "./memoize";
-export {debounce} from "../uikit/lib/util";
+import { identity, memoize } from "./memoize";
+import { IPReg } from "./ipreg";
+export { debounce } from "../uikit/lib/util";
 
 export class Promised {
   private promise: Promise<any>;
@@ -96,8 +97,72 @@ export const IS_WIN = typeof navigator !== "undefined" &&
 export const sanitizePath = identity(
   IS_WIN ? sanitizePathWindows : sanitizePathGeneric);
 
+export class PathInfo {
+  private baseField: string;
+
+  private extField: string;
+
+  private pathField: string;
+
+  private nameField: string;
+
+  private fullField: string;
+
+  constructor(base: string, ext: string, path: string) {
+    this.baseField = base;
+    this.extField = ext;
+    this.pathField = path;
+    this.update();
+  }
+
+  get base() {
+    return this.baseField;
+  }
+
+  set base(nv) {
+    this.baseField = sanitizePath(nv);
+    this.update();
+  }
+
+  get ext() {
+    return this.extField;
+  }
+
+  set ext(nv) {
+    this.extField = sanitizePath(nv);
+    this.update();
+  }
+
+  get name() {
+    return this.nameField;
+  }
+
+  get path() {
+    return this.pathField;
+  }
+
+  set path(nv) {
+    this.pathField = sanitizePath(nv);
+    this.update();
+  }
+
+  get full() {
+    return this.fullField;
+  }
+
+  private update() {
+    this.nameField = this.extField ? `${this.baseField}.${this.extField}` : this.baseField;
+    this.fullField = this.pathField ? `${this.pathField}/${this.nameField}` : this.nameField;
+  }
+
+  clone() {
+    return new PathInfo(this.baseField, this.extField, this.pathField);
+  }
+}
+
 // XXX cleanup + test
-export const parsePath = memoize(function parsePath(path: string | URL) {
+export const parsePath = memoize(function parsePath(
+    path: string | URL): PathInfo {
   if (path instanceof URL) {
     path = decodeURIComponent(path.pathname);
   }
@@ -127,13 +192,7 @@ export const parsePath = memoize(function parsePath(path: string | URL) {
   }
 
   path = pieces.join("/");
-  return {
-    path,
-    name,
-    base,
-    ext,
-    full: path ? `${path}/${name}` : name
-  };
+  return new PathInfo(base, ext, path);
 });
 
 export class CoalescedUpdate<T> extends Set<T> {
@@ -179,7 +238,10 @@ export interface URLd extends URL {
 Object.defineProperty(URL.prototype, "domain", {
   get() {
     try {
-      return hostToDomain(this.host) || this.host;
+      const {hostname} = this;
+      return IPReg.test(hostname) ?
+        hostname :
+        hostToDomain(hostname) || hostname;
     }
     catch (ex) {
       console.error(ex);
@@ -299,3 +361,20 @@ export function mapFilterInSitu<TRes, T>(
 export function randint(min: number, max: number) {
   return Math.floor(Math.random() * (max - min)) + min;
 }
+
+
+export function validateSubFolder(folder: string) {
+  if (!folder) {
+    return;
+  }
+  folder = folder.replace(/[/\\]+/g, "/");
+  if (folder.startsWith("/")) {
+    throw new Error("error.noabsolutepath");
+  }
+  if (/^[a-z]:\//i.test(folder)) {
+    throw new Error("error.noabsolutepath");
+  }
+  if (/^\.+\/|\/\.+\/|\/\.+$/g.test(folder)) {
+    throw new Error("error.nodotsinpath");
+  }
+}

lib/windowstatetracker.ts

@@ -3,6 +3,8 @@
 
 import { Prefs } from "./prefs";
 import { windows } from "./browser";
+// eslint-disable-next-line no-unused-vars
+import { Port } from "./bus";
 
 
 const VALID_WINDOW_STATES = Object.freeze(new Set(["normal", "maximized"]));
@@ -55,14 +57,16 @@ export class WindowStateTracker {
 
   getOptions(options: any) {
     const result = Object.assign(options, {
-      width: this.width,
-      height: this.height,
       state: this.state,
     });
+    if (result.state !== "maximized") {
+      result.width = this.width;
+      result.height = this.height;
       if (this.top >= 0) {
         result.top = this.top;
         result.left = this.left;
       }
+    }
     return result;
   }
 
@@ -78,6 +82,7 @@ export class WindowStateTracker {
     if (!this.windowId) {
       return;
     }
+    try {
       const window = await windows.get(this.windowId);
       if (!VALID_WINDOW_STATES.has(window.state)) {
         return;
@@ -95,17 +100,30 @@ export class WindowStateTracker {
       }
       await this.save();
     }
+    catch {
+      // ignored
+    }
+  }
 
-  track(windowId: number, port: any) {
+  track(windowId: number, port?: Port) {
     if (port) {
       port.on("resized", this.update);
+      port.on("unload", e => this.finalize(e));
+      port.on("disconnect", this.finalize.bind(this));
     }
     this.windowId = windowId;
   }
 
-  async finalize() {
+  async finalize(state?: any) {
+    if (state) {
+      this.left = state.left;
+      this.top = state.top;
+    }
     await this.update();
     this.windowId = 0;
+    if (state) {
+      await this.save();
+    }
   }
 
   async save() {

lib/windowutils.ts

@@ -1,30 +1,61 @@
 "use strict";
 // License: MIT
 
-import { windows, tabs, runtime } from "../lib/browser";
-import {getManager} from "./manager/man";
-import * as DEFAULT_ICONS from "../data/icons.json";
+import { windows, tabs, runtime, CHROME } from "../lib/browser";
+import { getManager } from "./manager/man";
+import DEFAULT_ICONS from "../data/icons.json";
+import { Prefs } from "./prefs";
+import { _ } from "./i18n";
+import { WindowStateTracker } from "./windowstatetracker";
+// eslint-disable-next-line no-unused-vars
+import { Port, Bus } from "./bus";
+import { timeout } from "./util";
 
 const DONATE_URL = "https://www.downthemall.org/howto/donate/";
+const DONATE_LANG_URLS = Object.freeze(new Map([
+  ["de", "https://www.downthemall.org/howto/donate/spenden/"],
+]));
 const MANAGER_URL = "/windows/manager.html";
 
-const IS_CHROME = navigator && navigator.userAgent.includes("Chrome");
-
-
-export async function mostRecentBrowser(): Promise<any> {
-  let window = Array.from(await windows.getAll({windowTypes: ["normal"]})).
-    filter((w: any) => w.type === "normal").pop();
+export async function mostRecentBrowser(incognito: boolean): Promise<any> {
+  let window;
+  try {
+    window = await windows.getCurrent();
+    if (window.type !== "normal") {
+      throw new Error("not a normal window");
+    }
+    if (incognito && !window.incognito) {
+      throw new Error("Not incognito");
+    }
+  }
+  catch {
+    try {
+      window = await windows.getLastFocused();
+      if (window.type !== "normal") {
+        throw new Error("not a normal window");
+      }
+      if (incognito && !window.incognito) {
+        throw new Error("Not incognito");
+      }
+    }
+    catch {
+      window = Array.from(await windows.getAll({windowTypes: ["normal"]})).
+        filter(
+          (w: any) => w.type === "normal" && !!w.incognito === !!incognito).
+        pop();
+    }
+  }
   if (!window) {
     window = await windows.create({
-      url: DONATE_URL,
+      incognito: !!incognito,
       type: "normal",
     });
   }
   return window;
 }
 
-export async function openInTab(url: string) {
-  const window = await mostRecentBrowser();
+export async function openInTab(url: string, incognito: boolean) {
+  const window = await mostRecentBrowser(incognito);
   await tabs.create({
     active: true,
     url,
@@ -33,7 +64,7 @@ export async function openInTab(url: string) {
   await windows.update(window.id, {focused: true});
 }
 
-export async function openInTabOrFocus(url: string) {
+export async function openInTabOrFocus(url: string, incognito: boolean) {
   const etabs = await tabs.query({
     url
   });
@@ -43,21 +74,22 @@ export async function openInTabOrFocus(url: string) {
     await windows.update(tab.windowId, {focused: true});
     return;
   }
-  await openInTab(url);
+  await openInTab(url, incognito);
 }
 
-export async function maybeOpenInTab(url: string) {
+export async function maybeOpenInTab(url: string, incognito: boolean) {
   const etabs = await tabs.query({
     url
   });
   if (etabs.length) {
     return;
   }
-  await openInTab(url);
+  await openInTab(url, incognito);
 }
 
 export async function donate() {
-  await openInTab(DONATE_URL);
+  const url = DONATE_LANG_URLS.get(_("language_code")) || DONATE_URL;
+  await openInTab(url, false);
 }
 
 export async function openPrefs() {
@@ -71,16 +103,64 @@ export async function openManager(focus = true) {
   catch (ex) {
     console.error(ex.toString(), ex);
   }
+  const url = runtime.getURL(MANAGER_URL);
+  const openInPopup = await Prefs.get("manager-in-popup");
+  if (openInPopup) {
+    const etabs = await tabs.query({
+      url
+    });
+    if (etabs.length) {
+      if (!focus) {
+        return;
+      }
+      const tab = etabs.pop();
+      await tabs.update(tab.id, {active: true});
+      await windows.update(tab.windowId, {focused: true});
+      return;
+    }
+
+    const tracker = new WindowStateTracker("manager", {
+      minWidth: 700,
+      minHeight: 500,
+    });
+    await tracker.init();
+    const windowOptions = tracker.getOptions({
+      url,
+      type: "popup",
+    });
+    const window = await windows.create(windowOptions);
+    tracker.track(window.id);
+    try {
+      if (!CHROME) {
+        windows.update(window.id, tracker.getOptions({}));
+      }
+      const port = await Promise.race<Port>([
+        new Promise<Port>(resolve => Bus.oncePort("manager", port => {
+          resolve(port);
+          return true;
+        })),
+        timeout<Port>(5 * 1000)]);
+      if (!port.isSelf) {
+        throw Error("Invalid sender connected");
+      }
+      tracker.track(window.id, port);
+    }
+    catch (ex) {
+      console.error("couldn't track manager", ex);
+    }
+
+    return;
+  }
   if (focus) {
-    await openInTabOrFocus(await runtime.getURL(MANAGER_URL));
+    await openInTabOrFocus(runtime.getURL(MANAGER_URL), false);
   }
   else {
-    await maybeOpenInTab(await runtime.getURL(MANAGER_URL));
+    await maybeOpenInTab(runtime.getURL(MANAGER_URL), false);
   }
 }
 
-export async function openUrls(urls: string) {
-  const window = await mostRecentBrowser();
+export async function openUrls(urls: string, incognito: boolean) {
+  const window = await mostRecentBrowser(incognito);
   for (const url of urls) {
     try {
       await tabs.create({
@@ -106,32 +186,10 @@ const ICONS = Object.freeze((() => {
   return new Map<string, string>(rv);
 })());
 
-let iconForPathPlatform: Function;
-if (IS_CHROME) {
-  const FOUR = 128;
-  const DOUBLE = 64;
-  iconForPathPlatform = function(icon: string, size: number) {
-    let scale = "1x";
-    if (size > FOUR) {
-      // wishful thinking at this point
-      scale = "4x";
-    }
-    else if (size > DOUBLE) {
-      scale = "2x";
-    }
-    return `chrome://fileicon/${icon}?scale=${scale}`;
-  };
-}
-else {
-  // eslint-disable-next-line no-unused-vars
-  iconForPathPlatform = function(icon: string, size: number) {
-    return ICONS.get(icon) || "icon-file-generic";
-  };
-}
+export const DEFAULT_ICON_SIZE = 16;
 
-
-// eslint-disable-next-line no-magic-numbers
-export function iconForPath(path: string, size = 16) {
+// eslint-disable-next-line no-unused-vars
+export function iconForPath(path: string, size = DEFAULT_ICON_SIZE) {
   const web = /^https?:\/\//.test(path);
   let file = path.split(/[\\/]/).pop();
   if (file) {
@@ -152,7 +210,7 @@ export function iconForPath(path: string, size = 16) {
       file = "file";
     }
   }
-  return iconForPathPlatform(file, size);
+  return ICONS.get(file) || "icon-file-generic";
 }
 
 /**

manifest.json

@@ -1,22 +1,21 @@
 {
   "manifest_version": 2,
-  "name": "DownThemAll!",
-  "version": "4.0.7",
+  "name": "TraitorousDownloading!",
+  "version": "4.2.6",
 
   "description": "__MSG_extensionDescription__",
-  "homepage_url": "https://downthemall.org/",
-  "author": "Nils Maier",
+  "homepage_url": "https://github.com/lordwelch/downthemall",
+  "author": "lordwelch",
 
   "default_locale": "en",
 
-  "content_security_policy": "script-src 'self'; style-src 'self' 'unsafe-inline'; default-src 'self'",
+  "content_security_policy": "script-src 'self'; style-src 'self' 'unsafe-inline'; img-src data: blob: 'self'; connect-src data: blob: http: https: 'self'; default-src 'self'",
 
   "icons": {
     "16": "style/icon16.png",
     "32": "style/icon32.png",
     "48": "style/icon48.png",
     "64": "style/icon64.png",
-    "96": "style/icon96.png",
     "128": "style/icon128.png",
     "256": "style/icon256.png"
   },
@@ -24,14 +23,20 @@
   "permissions": [
     "<all_urls>",
     "contextMenus",
-    "menus",
+    "cookies",
     "downloads",
     "downloads.open",
     "downloads.shelf",
+    "history",
+    "menus",
     "notifications",
+    "sessions",
     "storage",
     "tabs",
-    "webNavigation"
+    "theme",
+    "webNavigation",
+    "webRequest",
+    "webRequestBlocking"
   ],
 
   "background": {
@@ -49,11 +54,10 @@
       "32": "style/icon32.png",
       "48": "style/icon48.png",
       "64": "style/icon64.png",
-      "96": "style/icon96.png",
       "128": "style/icon128.png",
       "256": "style/icon256.png"
     },
-    "default_title": "DownThemAll!"
+    "default_title": "TraitorousDownloading!"
   },
 
   "options_ui": {
@@ -63,7 +67,7 @@
 
   "browser_specific_settings": {
     "gecko": {
-      "id": "dtalite@downthemall.org",
+      "id": "downloading@traitorousenterprises.net",
       "strict_min_version": "67.0"
     }
   }

package.json

@@ -1,13 +1,14 @@
 {
-  "name": "dtalite",
+  "name": "tdl",
   "version": "4.0.0",
-  "description": "DownThemAll! lite",
+  "description": "TraitorousDownloading!",
   "main": "main.js",
   "directories": {
     "lib": "lib"
   },
   "scripts": {
     "build": "util/build.py",
+    "build:cleanup": "rm -rf bundles",
     "build:bundles": "webpack",
     "build:regexps": "node util/makexregexps.js > data/xregexps.json",
     "stats": "cloc --vcs=git --exclude-lang=Markdown,SVG",
@@ -18,22 +19,26 @@
   "author": "Nils Maier",
   "license": "MIT",
   "devDependencies": {
-    "@types/node": "^12.7.2",
-    "@typescript-eslint/eslint-plugin": "^2.0.0",
-    "@typescript-eslint/parser": "^2.0.0",
+    "@types/node": "^12.7.8",
+    "@typescript-eslint/eslint-plugin": "^2.3.2",
+    "@typescript-eslint/parser": "^2.3.2",
     "chai": "^4.1.2",
-    "eslint": "^6.2.2",
-    "mocha": "^6.2.0",
-    "ts-loader": "^6.0.4",
-    "ts-node": "^8.3.0",
-    "typescript": "^3.5.3",
-    "webpack": "^4.39.3",
-    "webpack-cli": "^3.3.7",
+    "eslint": "^6.5.1",
+    "mocha": "^6.2.1",
+    "ts-loader": "^6.2.0",
+    "ts-node": "^8.4.1",
+    "typescript": "^3.6.3",
+    "webpack": "^4.41.0",
+    "webpack-cli": "^3.3.9",
     "xregexp": "^4.2.4"
   },
   "dependencies": {
     "@types/psl": "^1.1.0",
-    "psl": "^1.3.0",
-    "webextension-polyfill": "^0.4.0"
+    "@types/whatwg-mimetype": "^2.1.0",
+    "localforage": "^1.9.0",
+    "psl": "^1.4.0",
+    "webextension-polyfill": "^0.5.0",
+    "webextension-polyfill-ts": "^0.22.0",
+    "whatwg-mimetype": "^2.3.0"
   }
 }

scripts/gather.ts

@@ -77,6 +77,8 @@ function urlToUsable(e: any, u: string) {
 }
 
 class Gatherer {
+  private: boolean;
+
   textLinks: boolean;
 
   selectionOnly: boolean;
@@ -88,6 +90,7 @@ class Gatherer {
   transferable: string[];
 
   constructor(options: any) {
+    this.private = !!options.private;
     this.textLinks = options.textLinks;
     this.selectionOnly = options.selectionOnly;
     this.selection = options.selectionOnly ? getSelection() : null;
@@ -118,14 +121,26 @@ class Gatherer {
 
   *collectImageInternal(img: HTMLImageElement) {
     try {
-      const src = img.currentSrc || img.src;
+      {
+        const {src} = img;
         const item = this.makeItem(src, img);
         if (item) {
           item.fileName = "";
           item.description = item.title;
           yield item;
         }
+      }
+      {
+        const {currentSrc} = img;
+        const item = this.makeItem(currentSrc, img);
+        if (item) {
+          item.fileName = "";
+          item.description = item.title;
+          yield item;
+        }
+      }
 
+      {
         const {srcset} = img;
         if (!srcset) {
           return;
@@ -143,6 +158,7 @@ class Gatherer {
           }
         }
       }
+    }
     catch (ex) {
       console.error("oops image", ex.toString(), ex.stack, ex);
     }
@@ -255,6 +271,7 @@ class Gatherer {
       return {
         url: url.href,
         title,
+        private: this.private
       };
     }
     catch (ex) {
@@ -295,7 +312,7 @@ class Gatherer {
 function gather(msg: any, sender: any, callback: Function) {
   try {
     if (!msg || msg.type !== "DTA:gather" || !callback) {
-      return;
+      return Promise.resolve(null);
     }
     const gatherer = new Gatherer(msg);
     const result = {
@@ -313,10 +330,11 @@ function gather(msg: any, sender: any, callback: Function) {
       ),
     };
     urlToUsable(result, result.baseURL);
-    callback(result);
+    return Promise.resolve(result);
   }
   catch (ex) {
     console.error(ex.toString(), ex.stack, ex);
+    return Promise.resolve(null);
   }
 }
 

style/common.css

@@ -2,14 +2,29 @@
 /* License: gpl-v2 */
 
 :root {
+  --general-color: #2a2a2e;
+  --general-bgcolor: rgb(249, 249, 250);
+  --general-border-color: lightgray;
+  --general-input-color: black;
+  --general-input-bgcolor: white;
+  --general-button-color: black;
+  --general-button-bgcolor: rgb(246, 246, 246);
+  --general-button-bgcolor-hover: white;
+  --general-button-shadow: 0px 0px 5px 1px rgba(128, 128, 128, 0.5);
+  --menu-bgcolor: white;
+  --menu-bgcolor-hover: #2283fb;
+  --table-bgcolor: white;
+  --table-head-bgcolor: white;
   --toolbar-bg-color: rgb(248, 134, 6);
   --toolbar-active-border-color: #478de7;
   --toolbar-hover-border-color: red;
   --toolbar-hover-background: rgb(247, 149, 37);
   --toolbar-border-width: 2px;
+  --toolbar-border: 1px solid rgba(255, 255, 255, 0.5);
   --add-color: navy;
   --queue-color: gray;
   --pause-color: #ffa318;
+  --retry-color: rgb(0, 112, 204);
   --error-color: rgb(160, 13, 42);
   --running-color: #aae061;
   --finishing-color: #57cc12;
@@ -18,20 +33,82 @@
   --folder-color: rgb(214, 165, 4);
   --maskbutton-color: rgb(236, 185, 16);
   --missing-color: rgb(0, 82, 204);
+  --open-color: rgba(236, 185, 16, 0.8);
+  --status-icon-color: #363636;
+  --status-icon-color-hover: #6e6d6d;
+  --tile-url: url(tile.png);
+  --file-icon-image-color: rgb(17, 107, 163);
+  --popup-bgcolor: #fff;
+  --popup-color: #0c0c0d;
+  --modal-color: black;
+  --modal-bgcolor: white;
+}
+
+html.dark {
+  --add-color: lightblue;
+  --error-color: rgb(130, 3, 22);
+  --running-color: #67a041;
+  --finishing-color: #4bb111;
+  --done-color: #006f00;
+  --pause-color: #cf9308;
+  --general-bgcolor: #2a2a2e;
+  --general-border-color: rgb(85, 85, 85);
+  --general-button-bgcolor-hover: black;
+  --general-button-bgcolor: rgb(36, 36, 36);
+  --general-button-color: white;
+  --general-color: rgb(249, 249, 250);
+  --menu-bgcolor: black;
+  --menu-bgcolor-hover: #1a6bce;
+  --table-bgcolor: #1a1a1e;
+  --table-head-bgcolor: #3a3a3e;
+  --toolbar-bg-color: rgb(202, 108, 0);
+  --status-icon-color: #b9b9b9;
+  --status-icon-color-hover: #e2e2e2;
+  --tile-url: url(tile-dark.png?3);
+  --toolbar-border: 1px solid rgba(30, 30, 30, 0.5);
+  --file-icon-image-color: rgb(21, 130, 197);
+  --popup-bgcolor: #4a4a4f;
+  --popup-color: rgb(249, 249, 250);
+  --general-button-shadow: 0px 0px 7px 1px rgba(128, 128, 128, 0.8);
+  --modal-color: white;
+  --modal-bgcolor: #333;
+  scrollbar-color: rgba(249, 249, 250, 0.4) rgba(20, 20, 25, 0.3);
+}
+
+html.dark a {
+  color: lightblue;
+}
+
+html.dark ::-webkit-scrollbar {
+  background: rgba(20, 20, 25, 0.3);
+}
+
+html.dark ::-webkit-scrollbar-thumb {
+  background: rgba(249, 249, 250, 0.4);
+}
+
+html.dark ::-webkit-scrollbar-corner {
+  background: #000;
 }
 
 html[data-platform="mac"] {
   --folder-color: rgb(4, 102, 214);
 }
 
+html,
+body {
+  font-size: 10pt !important;
+}
+
 @font-face {
-  font-family: 'downthemall';
-  src: url('downthemall.woff2?75791791') format('woff2');
+  font-family: "downthemall";
+  src: url("downthemall.woff2?75791791") format("woff2");
   font-weight: normal;
   font-style: normal;
 }
 
-[class^="icon-"]:before, [class*=" icon-"]:before {
+[class^="icon-"]:before,
+[class*=" icon-"]:before {
   font-family: "downthemall";
   font-style: normal;
   font-weight: normal;
@@ -48,70 +125,187 @@ html[data-platform="mac"] {
   -moz-osx-font-smoothing: grayscale;
 }
 
-.icon-add:before { content: '\e800'; } /* '' */
-.icon-addsegment:before { content: '\e801'; } /* '' */
-.icon-bottom:before { content: '\e802'; } /* '' */
-.icon-picture:before { content: '\e803'; } /* '' */
-.icon-circle:before { content: '\e804'; } /* '' */
-.icon-delete:before { content: '\e805'; } /* '' */
-.icon-done:before { content: '\e806'; } /* '' */
-.icon-down:before { content: '\e807'; } /* '' */
-.icon-download:before { content: '\e808'; } /* '' */
-.icon-dupe:before { content: '\e809'; } /* '' */
-.icon-error:before { content: '\e80a'; } /* '' */
-.icon-failed:before { content: '\e80b'; } /* '' */
-.icon-file:before { content: '\e80c'; } /* '' */
-.icon-find:before { content: '\e80d'; } /* '' */
-.icon-folder:before { content: '\e80e'; } /* '' */
-.icon-force:before { content: '\e80f'; } /* '' */
-.icon-go:before { content: '\e810'; } /* '' */
-.icon-import:before { content: '\e811'; } /* '' */
-.icon-info:before { content: '\e812'; } /* '' */
-.icon-launch:before { content: '\e813'; } /* '' */
-.icon-missing:before { content: '\e814'; } /* '' */
-.icon-network-off:before { content: '\e815'; } /* '' */
-.icon-network-on:before { content: '\e816'; } /* '' */
-.icon-pause:before { content: '\e817'; } /* '' */
-.icon-remsegment:before { content: '\e818'; } /* '' */
-.icon-rename:before { content: '\e819'; } /* '' */
-.icon-save:before { content: '\e81a'; } /* '' */
-.icon-settings:before { content: '\e81b'; } /* '' */
-.icon-top:before { content: '\e81c'; } /* '' */
-.icon-unchecked:before { content: '\e81d'; } /* '' */
-.icon-unlimited:before { content: '\e81e'; } /* '' */
-.icon-link:before { content: '\e81f'; } /* '' */
-.icon-up:before { content: '\e820'; } /* '' */
-.icon-privacy:before { content: '\e821'; } /* '' */
-.icon-tags:before { content: '\e822'; } /* '' */
-.icon-attention:before { content: '\e823'; } /* '' */
-.icon-notification:before { content: '\e824'; } /* '' */
-.icon-file-video:before { content: '\e825'; } /* '' */
-.icon-file-generic:before { content: '\e826'; } /* '' */
-.icon-question-dark:before { content: '\e827'; } /* '' */
-.icon-filter:before { content: '\f0b0'; } /* '' */
-.icon-donate:before { content: '\f0d6'; } /* '' */
-.icon-file-doc:before { content: '\f0f6'; } /* '' */
-.icon-interface:before { content: '\f108'; } /* '' */
-.icon-folder-1:before { content: '\f115'; } /* '' */
-.icon-sort-asc:before { content: '\f15d'; } /* '' */
-.icon-sort-desc:before { content: '\f15e'; } /* '' */
-.icon-file-pdf:before { content: '\f1c1'; } /* '' */
-.icon-file-word:before { content: '\f1c2'; } /* '' */
-.icon-file-image:before { content: '\f1c5'; } /* '' */
-.icon-file-archive:before { content: '\f1c6'; } /* '' */
-.icon-file-audio:before { content: '\f1c7'; } /* '' */
-.icon-toggle:before { content: '\f205'; } /* '' */
-.icon-server:before { content: '\f233'; } /* '' */
-.icon-question-light:before { content: '\f29c'; } /* '' */
+.icon-add:before {
+  content: "\e800";
+} /* '' */
+.icon-addsegment:before {
+  content: "\e801";
+} /* '' */
+.icon-bottom:before {
+  content: "\e802";
+} /* '' */
+.icon-picture:before {
+  content: "\e803";
+} /* '' */
+.icon-circle:before {
+  content: "\e804";
+} /* '' */
+.icon-delete:before {
+  content: "\e805";
+} /* '' */
+.icon-done:before {
+  content: "\e806";
+} /* '' */
+.icon-down:before {
+  content: "\e807";
+} /* '' */
+.icon-download:before {
+  content: "\e808";
+} /* '' */
+.icon-dupe:before {
+  content: "\e809";
+} /* '' */
+.icon-error:before {
+  content: "\e80a";
+} /* '' */
+.icon-failed:before {
+  content: "\e80b";
+} /* '' */
+.icon-file:before {
+  content: "\e80c";
+} /* '' */
+.icon-find:before {
+  content: "\e80d";
+} /* '' */
+.icon-folder:before {
+  content: "\e80e";
+} /* '' */
+.icon-force:before {
+  content: "\e80f";
+} /* '' */
+.icon-go:before {
+  content: "\e810";
+} /* '' */
+.icon-import:before {
+  content: "\e811";
+} /* '' */
+.icon-info:before {
+  content: "\e812";
+} /* '' */
+.icon-launch:before {
+  content: "\e813";
+} /* '' */
+.icon-missing:before {
+  content: "\e814";
+} /* '' */
+.icon-network-off:before {
+  content: "\e815";
+} /* '' */
+.icon-network-on:before {
+  content: "\e816";
+} /* '' */
+.icon-pause:before {
+  content: "\e817";
+} /* '' */
+.icon-remsegment:before {
+  content: "\e818";
+} /* '' */
+.icon-rename:before {
+  content: "\e819";
+} /* '' */
+.icon-save:before {
+  content: "\e81a";
+} /* '' */
+.icon-settings:before {
+  content: "\e81b";
+} /* '' */
+.icon-top:before {
+  content: "\e81c";
+} /* '' */
+.icon-unchecked:before {
+  content: "\e81d";
+} /* '' */
+.icon-unlimited:before {
+  content: "\e81e";
+} /* '' */
+.icon-link:before {
+  content: "\e81f";
+} /* '' */
+.icon-up:before {
+  content: "\e820";
+} /* '' */
+.icon-privacy:before {
+  content: "\e821";
+} /* '' */
+.icon-tags:before {
+  content: "\e822";
+} /* '' */
+.icon-attention:before {
+  content: "\e823";
+} /* '' */
+.icon-notification:before {
+  content: "\e824";
+} /* '' */
+.icon-file-video:before {
+  content: "\e825";
+} /* '' */
+.icon-file-generic:before {
+  content: "\e826";
+} /* '' */
+.icon-question-dark:before {
+  content: "\e827";
+} /* '' */
+.icon-forward:before {
+  content: "\e828";
+} /* '' */
+.icon-filter:before {
+  content: "\f0b0";
+} /* '' */
+.icon-donate:before {
+  content: "\f0d6";
+} /* '' */
+.icon-file-doc:before {
+  content: "\f0f6";
+} /* '' */
+.icon-interface:before {
+  content: "\f108";
+} /* '' */
+.icon-folder-1:before {
+  content: "\f115";
+} /* '' */
+.icon-sort-asc:before {
+  content: "\f15d";
+} /* '' */
+.icon-sort-desc:before {
+  content: "\f15e";
+} /* '' */
+.icon-file-pdf:before {
+  content: "\f1c1";
+} /* '' */
+.icon-file-word:before {
+  content: "\f1c2";
+} /* '' */
+.icon-file-image:before {
+  content: "\f1c5";
+} /* '' */
+.icon-file-archive:before {
+  content: "\f1c6";
+} /* '' */
+.icon-file-audio:before {
+  content: "\f1c7";
+} /* '' */
+.icon-toggle-off:before {
+  content: "\f204";
+} /* '' */
+.icon-toggle-on:before {
+  content: "\f205";
+} /* '' */
+.icon-server:before {
+  content: "\f233";
+} /* '' */
+.icon-question-light:before {
+  content: "\f29c";
+} /* '' */
 
 @media (min-resolution: 144dpi) {
-  [class^="icon-file-"]:before, [class*=" icon-file-"]:before {
+  [class^="icon-file-"]:before,
+  [class*=" icon-file-"]:before {
     font-weight: bold !important;
   }
 }
 
 .icon-file-image {
-  color: rgb(17, 107, 163);
+  color: var(--file-icon-image-color);
 }
 
 .icon-file-pdf,
@@ -132,18 +326,29 @@ html[data-platform="mac"] {
   color: rgb(202, 81, 198);
 }
 
-body, html {
-  background: #F6F6F8;
-  color: #0C0C0D;
+body,
+html {
   font: message-box;
-  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Ubuntu', 'Helvetica Neue', sans-serif;
+  font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", "Ubuntu",
+    "Helvetica Neue", sans-serif;
   height: 100%;
   margin: 0;
   padding: 0;
   width: 100%;
+  color: var(--general-color);
+  background: var(--general-bgcolor);
 }
 
-h1, h2, h3, h4 {
+html#popup,
+html#popup > body {
+  color: var(--popup-color);
+  background: var(--popup-bgcolor);
+}
+
+h1,
+h2,
+h3,
+h4 {
   font: caption;
   font-weight: bold;
 }
@@ -183,7 +388,11 @@ section {
 }
 
 .virtualtable-column:active {
-  background-image: linear-gradient(to top, rgba(0,0,0,0.03), rgba(128,128,128,0.1));
+  background-image: linear-gradient(
+    to top,
+    rgba(0, 0, 0, 0.03),
+    rgba(128, 128, 128, 0.1)
+  );
 }
 
 th.virtualtable {
@@ -210,8 +419,12 @@ td.virtualtable {
   font-size: 12px;
   align-items: stretch;
   justify-items: center;
-  background: linear-gradient(to bottom, rgba(128,128,128,0.1) 0%,rgba(0,0,0,0) 100%);
-  border-top: 1px solid rgba(128,128,128,0.6);
+  background: linear-gradient(
+    to bottom,
+    rgba(128, 128, 128, 0.1) 0%,
+    rgba(0, 0, 0, 0) 100%
+  );
+  border-top: 1px solid rgba(128, 128, 128, 0.6);
   display: flex;
   margin-bottom: 1ex;
   overflow: auto;
@@ -238,15 +451,15 @@ td.virtualtable {
   flex-grow: 3;
   margin-right: 2ex;
   padding-right: 1ex;
-  border-right: 1px dotted rgba(128,128,128,0.6);
+  border-right: 1px dotted rgba(128, 128, 128, 0.6);
 }
 
 #statusPrefs {
   cursor: pointer;
-  color: #363636;
+  color: var(--status-icon-color);
 }
 #statusPrefs:hover {
-  color: #6e6d6d;
+  color: var(--status-icon-color-hover);
 }
 
 .dropdown {
@@ -264,13 +477,14 @@ td.virtualtable {
   outline: none;
   position: absolute;
   top: 0;
-  width:100%;
+  width: 100%;
 }
 
 .dropdown input {
   -moz-appearance: none;
   -webkit-appearance: none;
   appearance: none;
+  color: black;
   background: white;
   border: none;
   bottom: 2px;
@@ -293,7 +507,6 @@ td.virtualtable {
   padding-bottom: 1ex;
 }
 
-
 @supports (not (-moz-appearance: none)) {
   .dropdown select {
     background: white;
@@ -363,3 +576,54 @@ td.virtualtable {
 #maskButton {
   color: var(--maskbutton-color);
 }
+
+table.virtualtable,
+.virtualtable-body {
+  color: var(--general-color);
+  background: var(--table-bgcolor);
+}
+.virtualtable-head,
+.virtualtable-head > table {
+  background: var(--table-head-bgcolor) !important;
+}
+
+.virtualtable-column {
+  border-right: 1px solid var(--general-border-color);
+}
+
+.virtualtable-cell {
+  border-right: 1px dotted var(--general-border-color);
+}
+
+.virtualtable-head,
+.virtualtable-body {
+  border-bottom: 1px solid var(--general-border-color);
+}
+
+ul.context-menu,
+ul.context-menu ul {
+  color: var(--general-color);
+  background: var(--menu-bgcolor);
+}
+
+.context-menu-item:hover:not(.context-menu-seperator),
+.context-menu-item:hover:not(.context-menu-seperator) > * {
+  background: var(--menu-bgcolor-hover);
+}
+
+html.dark .context-menu-item.disabled,
+html.dark .context-menu-item.disabled > * {
+  opacity: 0.7;
+}
+
+input {
+  color: var(--general-input-color);
+  background: var(--general-input-bgcolor);
+  border: inherit;
+  padding: 2px;
+}
+
+.modal-dialog {
+  color: var(--modal-color);
+  background: var(--modal-bgcolor);
+}

style/manager.css

@@ -16,7 +16,7 @@ body > * {
 #toolbar {
   display: flex;
   margin: 0;
-  background: var(--toolbar-bg-color) url(tile.png) repeat-x;
+  background: var(--toolbar-bg-color) var(--tile-url) repeat-x;
 }
 
 #toolbar .spacer {
@@ -42,9 +42,9 @@ body > * {
   justify-content: center;
   align-items: center;
   cursor: pointer;
-  box-shadow: 0px 0px 5px 1px rgba(128,128,128,0.5);
-  background: rgb(246,246,246);
-  color: black;
+  box-shadow: var(--general-button-shadow);
+  background: var(--general-button-bgcolor);
+  color: var(--general-button-color);
   transition: box-shadow 0.5s, background 1s;
   font-size: 24px;
   line-height: 24px;
@@ -60,7 +60,7 @@ body > * {
 }
 
 #toolbar > .button:hover:not(.disabled) {
-  background: white;
+  background: var(--general-button-bgcolor-hover);
   box-shadow: 0px 0px 7px 2px rgba(70,70,70,0.75);
 }
 
@@ -88,14 +88,14 @@ body > * {
   flex: 1 1 auto;
   overflow: hidden;
   display: grid;
-  background: white;
+  background: var(--general-bgcolor);
 }
 
 #loading {
   display: flex;
   justify-content: center;
   align-items: center;
-  background: rgba(255,255,255,0.9);
+  background: var(--general-button-bgcolor);
   font-weight: bolder;
   font-size: 200%;
   z-index: 10;
@@ -108,11 +108,11 @@ body > * {
 }
 
 #colURL {
-  width: 38%;
+  width: 42%;
 }
 
 #colPercent {
-  width: 3em;
+  width: 4em;
   min-width: 3em;
 }
 
@@ -121,11 +121,11 @@ body > * {
 }
 
 #colSize {
-  width: 15em;
+  width: 14em;
 }
 
 #colSpeed {
-  width: 6em;
+  width: 7em;
 }
 
 #colDomain,
@@ -154,6 +154,14 @@ body > * {
   height: 26px;
 }
 
+.virtualtable-row.opening {
+  background: var(--open-color) !important;
+}
+
+.virtualtable-progress-container {
+  border-radius: 2px;
+}
+
 .virtualtable-progress-bar {
   height: 14px;
 }
@@ -194,6 +202,23 @@ body > * {
   );
 }
 
+.retrying .virtualtable-column-2 .virtualtable-icon {
+  color: var(--retry-color);
+}
+.retrying .virtualtable-column-2 .virtualtable-progress-bar {
+  background: var(--retry-color);
+}
+.retrying .virtualtable-column-2 .virtualtable-progress-undetermined {
+  background: repeating-linear-gradient(
+    45deg,
+    var(--retry-color),
+    var(--retry-color) 6px,
+    transparent 6px,
+    transparent 12px
+  );
+}
+
+
 .missing .virtualtable-column-2 .virtualtable-icon,
 .canceled .virtualtable-column-2 .virtualtable-icon {
   color: var(--error-color);
@@ -262,6 +287,7 @@ body > * {
 }
 
 .virtualtable-column-6,
+.virtualtable-column-4,
 .virtualtable-column-3 {
   text-align: right;
 }
@@ -285,7 +311,7 @@ body > * {
   color: crimson;
 }
 #statusNetwork.icon-network-on {
-  color: navy;
+  color: var(--add-color);
 }
 
 #statusFilter {
@@ -318,6 +344,7 @@ body > * {
   height: 16px;
   -moz-appearance: none;
   border: 0;
+  outline: 0;
   background: transparent;
   width: calc(100% - 28px);
 }
@@ -377,7 +404,7 @@ body > * {
   font-size: 10pt !important;
 }
 #nagging {
-  border-top: 1px solid lightgray;
+  border-top: 1px solid var(--general-border-color);
   display: grid;
   grid-template-columns: 1fr auto auto auto;
   align-content: center;
@@ -430,6 +457,8 @@ body > * {
   justify-items: stretch;
   border-radius: 4px;
   box-shadow: 2px 2px 6px black;
+  -webkit-user-select: none;
+  user-select: none;
 }
 
 #tooltip-infos {
@@ -442,8 +471,13 @@ body > * {
 }
 
 #tooltip-icon {
-  font-size: 48px;
-  line-height: 48px;
+  height: 64px;
+  width: 64px;
+  background-size: 64px 64px;
+  background-repeat: no-repeat;
+  background-position: center center;
+  font-size: 64px;
+  line-height: 64px;
   padding: 6px;
   text-align: center;
   grid-row: 1/-1;
@@ -496,3 +530,23 @@ body > * {
   width: 100%;
   background: var(--done-color);
 }
+
+#tooltip-eta.single {
+  font-weight: bold;
+  grid-column-end: span 2;
+}
+
+.deletefiles-list {
+  padding-left: 1ex;
+  padding-right: 1.5ex;
+  border: 1px solid lightgray;
+  border-radius: 6px;
+  background-color: rgba(128,128,128,0.1);
+  max-height: 8em;
+  overflow-y: auto;
+}
+.deletefiles-list > li {
+  list-style-type: none;
+  padding: 0;
+  margin: 0;
+}

style/prefs.css

@@ -1,8 +1,8 @@
 /* License: gpl-v2 */
-@import 'common.css';
+@import "common.css";
 
-html, body {
-  background: transparent !important;
+html,
+body {
   width: 100%;
   height: 100%;
 }
@@ -19,9 +19,10 @@ article {
 
 #tabs {
   display: flex;
-  background: url(icon64.png) 1em 50%/32px 32px no-repeat, url(tile.png) repeat-x, var(--toolbar-bg-color);
+  background: url(icon64.png) 1em 50%/32px 32px no-repeat,
+    var(--tile-url) repeat-x, var(--toolbar-bg-color);
   padding-left: calc(2em + 32px);
-  color: white;
+  color: var(--general-bgcolor);
 }
 
 input.tab {
@@ -54,9 +55,10 @@ input.tab {
 #tabsel-general:checked ~ #tabs #tabel-general,
 #tabsel-filters:checked ~ #tabs #tabel-filters,
 #tabsel-network:checked ~ #tabs #tabel-network {
-  color: black !important;
-  background: white;
-  border-top: var(--toolbar-border-width) solid var(--toolbar-active-border-color);
+  color: var(--general-color) !important;
+  background: var(--general-bgcolor);
+  border-top: var(--toolbar-border-width) solid
+    var(--toolbar-active-border-color);
 }
 
 #tabs > label {
@@ -64,13 +66,14 @@ input.tab {
   border-top: var(--toolbar-border-width) solid transparent;
   border-left: 1px solid transparent;
   border-right: 1px solid transparent;
-  border-left: 1px solid rgba(255, 255, 255, 0.5);
-  border-right: 1px solid rgba(255, 255, 255, 0.5);
+  border-left: var(--toolbar-border);
+  border-right: var(--toolbar-border);
   background: var(--toolbar-bg-color);
 }
 
 #tabs > label:hover:not(:checked) {
-  border-top: var(--toolbar-border-width) solid var(--toolbar-hover-border-color);
+  border-top: var(--toolbar-border-width) solid
+    var(--toolbar-hover-border-color);
   background: var(--toolbar-hover-background);
 }
 
@@ -102,7 +105,7 @@ input.tab {
   text-align: center;
 }
 
-.buttons > button{
+.buttons > button {
   margin: 0 2em;
 }
 
@@ -113,15 +116,27 @@ input.tab {
 fieldset {
   display: flex;
   margin-bottom: 1em;
-  border: 1px solid lightgray;
+  border: 1px solid var(--general-border-color);
   border-radius: 6px;
-  box-shadow: 1px 1px 6px lightgray;
+  box-shadow: 1px 1px 6px var(--general-border-color);
   background: rgba(128, 128, 128, 0.05);
   flex-direction: column;
   max-width: 60em;
   padding: 1.2em;
 }
 
+.optiongroups,
+fieldset > label {
+  display: flex;
+  align-items: center;
+}
+
+fieldset > label > input,
+fieldset > label > select {
+  margin-left: 1ex;
+  margin-right: 1ex;
+}
+
 legend {
   font-weight: bold;
   font-size: 120%;
@@ -134,8 +149,36 @@ legend {
 }
 
 .virtualtable-container {
-  border: 1px solid lightgray;
+  border: 1px solid var(--general-border-color);
   border-radius: 6px;
   background: rgba(128, 128, 128, 0.05);
-  box-shadow: 1px 1px 6px lightgray;
+  box-shadow: 1px 1px 6px var(--general-border-color);
+}
+
+#network-general {
+  display: grid;
+  grid-template-columns: auto 1fr;
+  grid-column-gap: 1em;
+  grid-row-gap: 1ex;
+}
+
+.optiongroups {
+  display: grid;
+  grid-template-columns: auto 1fr;
+  grid-column-gap: 1em;
+  grid-row-gap: 1ex;
+}
+
+.optiongroups > div,
+.optiongroups > div > label {
+  display: flex;
+  align-items: center;
+}
+.optiongroups input {
+  margin-left: 1em;
+  margin-right: 0.7ex;
+}
+
+hr {
+  width: 100%;
 }

style/select.css

@@ -107,7 +107,7 @@ body > * {
   padding: 0;
   padding-left: calc(2em + 32px);
   color: black;
-  background: url(icon32.png) 1em 0/32px 32px no-repeat, url(tile.png) repeat-x, var(--toolbar-bg-color);
+  background: url(icon32.png) 1em 0/32px 32px no-repeat, var(--tile-url) repeat-x, var(--toolbar-bg-color);
   font: caption;
   font-size: 150%;
   font-weight: bold;
@@ -117,7 +117,7 @@ body > * {
 }
 @media (-webkit-min-device-pixel-ratio: 1.3), (min-resolution: 124.8dpi) {
   #tabs {
-    background: url(icon64.png) 1em 50%/32px 32px no-repeat, url(tile.png) repeat-x, var(--toolbar-bg-color);
+    background: url(icon64.png) 1em 50%/32px 32px no-repeat, var(--tile-url) repeat-x, var(--toolbar-bg-color);
   }
 }
 
@@ -145,27 +145,27 @@ body > * {
   -webkit-user-select: none;
   user-select: none;
   background: var(--toolbar-bg-color);
-  color: white;
+  color: var(--general-color);
   min-width: 10em;
   padding: 1ex;
   padding-left: 1em;
   cursor: pointer;
   border: 0;
   border-top: var(--toolbar-border-width) solid transparent;
-  border-left: 1px solid rgba(255,255,255,0.3);
-  border-right: 1px solid rgba(255,255,255,0.3);
+  border-left: var(--toolbar-border);
+  border-right: var(--toolbar-border);
   transition: border 1s;
 }
 
 .tab:not(.active):not(.disabled):hover {
   border-top: var(--toolbar-border-width) solid var(--toolbar-hover-border-color);
-  color: rgb(255, 226, 167);
+  color: var(--general-color);
   background: var(--toolbar-hover-background);
 }
 
 .tab.active {
-  color: black;
-  background: white;
+  color: var(--general-color);
+  background: var(--table-head-bgcolor);
   border-top: var(--toolbar-border-width) solid var(--toolbar-active-border-color);
   border-left: 1px solid transparent;
   border-right: 1px solid transparent;
@@ -232,3 +232,7 @@ body > * {
 #maskButton {
   justify-self: flex-start;
 }
+
+#btnDownload {
+  font-weight: bold;
+}

style/single.css

@@ -63,7 +63,8 @@ p.example {
   align-items: center;
 }
 
-#options > #maskOptions {
+#options > #subfolderOptions,
+#options > #maskOptions, #options > #serverOptions {
   display: grid;
   grid-template-columns: 2fr auto auto;
 }
@@ -81,3 +82,7 @@ h3 {
   font-weight: normal;
   font-style: italic;
 }
+
+#btnDownload {
+  font-weight: bold;
+}

tests/.editorconfig

@@ -0,0 +1,10 @@
+root = true
+
+[*]
+charset = utf-8
+end_of_line = lf
+indent_size = 2
+indent_style = space
+insert_final_newline = true
+tab_width = 2
+trim_trailing_whitespace = true

tests/test_batches.js

@@ -113,4 +113,29 @@ describe("BatchGenerator", function() {
     expect(items[0]).to.equal(gen.preview);
     expect(gen.hasInvalid).to.be.true;
   });
+
+  it("characters", function() {
+    const gen = new BatchGenerator("abc[a:c].lol[1].b");
+    const items = Array.from(gen);
+    expect(items).to.deep.equal([
+      "abca.lol[1].b",
+      "abcb.lol[1].b",
+      "abcc.lol[1].b",
+    ]);
+    expect(items.length).to.equal(gen.length);
+    expect(items[0]).to.equal(gen.preview);
+  });
+
+  it("characters two", function() {
+    const gen = new BatchGenerator("abc[D:G].lol[1].b");
+    const items = Array.from(gen);
+    expect(items).to.deep.equal([
+      "abcD.lol[1].b",
+      "abcE.lol[1].b",
+      "abcF.lol[1].b",
+      "abcG.lol[1].b",
+    ]);
+    expect(items.length).to.equal(gen.length);
+    expect(items[0]).to.equal(gen.preview);
+  });
 });

tests/test_cdheaderparser.js

@@ -0,0 +1,289 @@
+/* eslint-disable max-len */
+/* eslint-env node */
+"use strict";
+// License: MPL-2
+
+// eslint-disable-next-line @typescript-eslint/no-var-requires
+const { CDHeaderParser } = require("../lib/cdheaderparser");
+
+const parser = new CDHeaderParser();
+
+function check(header, expected) {
+  expect(parser.parse(header)).to.equal(expected);
+}
+
+function nocheck(header, expected) {
+  expect(parser.parse(header)).not.to.equal(expected);
+}
+
+describe("CDHeaderParser", function() {
+  it("parse wget", function() {
+    // From wget, test_parse_content_disposition
+    // http://git.savannah.gnu.org/cgit/wget.git/tree/src/http.c?id=8551ceccfedb4390fbfa82c12f0ff714dab1ac76#n5325
+    check("filename=\"file.ext\"", "file.ext");
+    check("attachment; filename=\"file.ext\"", "file.ext");
+    check("attachment; filename=\"file.ext\"; dummy", "file.ext");
+    check("attachment", ""); // wget uses NULL, we use "".
+    check("attachement; filename*=UTF-8'en-US'hello.txt", "hello.txt");
+    check("attachement; filename*0=\"hello\"; filename*1=\"world.txt\"",
+      "helloworld.txt");
+    check("attachment; filename=\"A.ext\"; filename*=\"B.ext\"", "B.ext");
+    check("attachment; filename*=\"A.ext\"; filename*0=\"B\"; filename*1=\"B.ext\"",
+      "A.ext");
+    // This test is faulty - https://savannah.gnu.org/bugs/index.php?52531
+    //check("filename**0=\"A\"; filename**1=\"A.ext\"; filename*0=\"B\";filename*1=\"B\"", "AA.ext");
+  });
+
+  it("parse Firefox", function() {
+    // From Firefox
+    // https://searchfox.org/mozilla-central/rev/45a3df4e6b8f653b0103d18d97c34dd666706358/netwerk/test/unit/test_MIME_params.js
+    // Changed as follows:
+    // - Replace error codes with empty string (we never throw).
+
+    const BS = "\\";
+    const DQUOTE = "\"";
+    // No filename parameter: return nothing
+    check("attachment;", "");
+    // basic
+    check("attachment; filename=basic", "basic");
+    // extended
+    check("attachment; filename*=UTF-8''extended", "extended");
+    // prefer extended to basic (bug 588781)
+    check("attachment; filename=basic; filename*=UTF-8''extended", "extended");
+    // prefer extended to basic (bug 588781)
+    check("attachment; filename*=UTF-8''extended; filename=basic", "extended");
+    // use first basic value (invalid; error recovery)
+    check("attachment; filename=first; filename=wrong", "first");
+    // old school bad HTTP servers: missing 'attachment' or 'inline'
+    // (invalid; error recovery)
+    check("filename=old", "old");
+    check("attachment; filename*=UTF-8''extended", "extended");
+    // continuations not part of RFC 5987 (bug 610054)
+    check("attachment; filename*0=foo; filename*1=bar", "foobar");
+    // Return first continuation (invalid; error recovery)
+    check("attachment; filename*0=first; filename*0=wrong; filename=basic", "first");
+    // Only use correctly ordered continuations  (invalid; error recovery)
+    check("attachment; filename*0=first; filename*1=second; filename*0=wrong", "firstsecond");
+    // prefer continuation to basic (unless RFC 5987)
+    check("attachment; filename=basic; filename*0=foo; filename*1=bar", "foobar");
+    // Prefer extended to basic and/or (broken or not) continuation
+    // (invalid; error recovery)
+    check("attachment; filename=basic; filename*0=first; filename*0=wrong; filename*=UTF-8''extended", "extended");
+    // RFC 2231 not clear on correct outcome: we prefer non-continued extended
+    // (invalid; error recovery)
+    check("attachment; filename=basic; filename*=UTF-8''extended; filename*0=foo; filename*1=bar", "extended");
+    // Gaps should result in returning only value until gap hit
+    // (invalid; error recovery)
+    check("attachment; filename*0=foo; filename*2=bar", "foo");
+    // Don't allow leading 0's (*01) (invalid; error recovery)
+    check("attachment; filename*0=foo; filename*01=bar", "foo");
+    // continuations should prevail over non-extended (unless RFC 5987)
+    check("attachment; filename=basic; filename*0*=UTF-8''multi;\r\n" +
+      " filename*1=line;\r\n" +
+      " filename*2*=%20extended",
+    "multiline extended");
+    // Gaps should result in returning only value until gap hit
+    // (invalid; error recovery)
+    check("attachment; filename=basic; filename*0*=UTF-8''multi;\r\n" +
+      " filename*1=line;\r\n" +
+      " filename*3*=%20extended",
+    "multiline");
+    // First series, only please, and don't slurp up higher elements (*2 in this
+    // case) from later series into earlier one (invalid; error recovery)
+    check("attachment; filename=basic; filename*0*=UTF-8''multi;\r\n" +
+      " filename*1=line;\r\n" +
+      " filename*0*=UTF-8''wrong;\r\n" +
+      " filename*1=bad;\r\n" +
+      " filename*2=evil",
+    "multiline");
+    // RFC 2231 not clear on correct outcome: we prefer non-continued extended
+    // (invalid; error recovery)
+    check("attachment; filename=basic; filename*0=UTF-8''multi\r\n;" +
+      " filename*=UTF-8''extended;\r\n" +
+      " filename*1=line;\r\n" +
+      " filename*2*=%20extended",
+    "extended");
+    // sneaky: if unescaped, make sure we leave UTF-8'' in value
+    check("attachment; filename*0=UTF-8''unescaped;\r\n" +
+      " filename*1*=%20so%20includes%20UTF-8''%20in%20value",
+    "UTF-8''unescaped so includes UTF-8'' in value");
+    // sneaky: if unescaped, make sure we leave UTF-8'' in value
+    check("attachment; filename=basic; filename*0=UTF-8''unescaped;\r\n" +
+      " filename*1*=%20so%20includes%20UTF-8''%20in%20value",
+    "UTF-8''unescaped so includes UTF-8'' in value");
+    // Prefer basic over invalid continuation
+    // (invalid; error recovery)
+    check("attachment; filename=basic; filename*1=multi;\r\n" +
+      " filename*2=line;\r\n" +
+      " filename*3*=%20extended",
+    "basic");
+    // support digits over 10
+    check("attachment; filename=basic; filename*0*=UTF-8''0;\r\n" +
+      " filename*1=1; filename*2=2;filename*3=3;filename*4=4;filename*5=5;\r\n" +
+      " filename*6=6; filename*7=7;filename*8=8;filename*9=9;filename*10=a;\r\n" +
+      " filename*11=b; filename*12=c;filename*13=d;filename*14=e;filename*15=f\r\n",
+    "0123456789abcdef");
+    // support digits over 10 (detect gaps)
+    check("attachment; filename=basic; filename*0*=UTF-8''0;\r\n" +
+      " filename*1=1; filename*2=2;filename*3=3;filename*4=4;filename*5=5;\r\n" +
+      " filename*6=6; filename*7=7;filename*8=8;filename*9=9;filename*10=a;\r\n" +
+      " filename*11=b; filename*12=c;filename*14=e\r\n",
+    "0123456789abc");
+    // return nothing: invalid
+    // (invalid; error recovery)
+    check("attachment; filename*1=multi;\r\n" +
+      " filename*2=line;\r\n" +
+      " filename*3*=%20extended",
+    "");
+    // Bug 272541: Empty disposition type treated as "attachment"
+    // sanity check
+    check("attachment; filename=foo.html", "foo.html");
+    // the actual bug
+    check("; filename=foo.html", "foo.html");
+    // regression check, but see bug 671204
+    check("filename=foo.html", "foo.html");
+    // Bug 384571: RFC 2231 parameters not decoded when appearing in reversed order
+    // check ordering
+    check("attachment; filename=basic; filename*0*=UTF-8''0;\r\n" +
+      " filename*1=1; filename*2=2;filename*3=3;filename*4=4;filename*5=5;\r\n" +
+      " filename*6=6; filename*7=7;filename*8=8;filename*9=9;filename*10=a;\r\n" +
+      " filename*11=b; filename*12=c;filename*13=d;filename*15=f;filename*14=e;\r\n",
+    "0123456789abcdef");
+    // check non-digits in sequence numbers
+    check("attachment; filename=basic; filename*0*=UTF-8''0;\r\n" +
+      " filename*1a=1\r\n",
+    "0");
+    // check duplicate sequence numbers
+    check("attachment; filename=basic; filename*0*=UTF-8''0;\r\n" +
+      " filename*0=bad; filename*1=1;\r\n",
+    "0");
+    // check overflow
+    check("attachment; filename=basic; filename*0*=UTF-8''0;\r\n" +
+      " filename*11111111111111111111111111111111111111111111111111111111111=1",
+    "0");
+    // check underflow
+    check("attachment; filename=basic; filename*0*=UTF-8''0;\r\n" +
+      " filename*-1=1",
+    "0");
+    // check mixed token/quoted-string
+    check("attachment; filename=basic; filename*0=\"0\";\r\n" +
+      " filename*1=1;\r\n" +
+      " filename*2*=%32",
+    "012");
+    // check empty sequence number
+    check("attachment; filename=basic; filename**=UTF-8''0\r\n", "basic");
+    // Bug 419157: ensure that a MIME parameter with no charset information
+    // fallbacks to Latin-1
+    check("attachment;filename=IT839\x04\xB5(m8)2.pdf;", "IT839\u0004\u00b5(m8)2.pdf");
+    // Bug 588389: unescaping backslashes in quoted string parameters
+    // '\"', should be parsed as '"'
+    check(`attachment; filename=${DQUOTE}${BS + DQUOTE}${DQUOTE}`, DQUOTE);
+    // 'a\"b', should be parsed as 'a"b'
+    check(`attachment; filename=${DQUOTE}a${BS + DQUOTE}b${DQUOTE}`, `a${DQUOTE}b`);
+    // '\x', should be parsed as 'x'
+    check(`attachment; filename=${DQUOTE}${BS}x${DQUOTE}`, "x");
+    // test empty param (quoted-string)
+    check(`attachment; filename=${DQUOTE}${DQUOTE}`, "");
+    // test empty param
+    check("attachment; filename=", "");
+    // Bug 601933: RFC 2047 does not apply to parameters (at least in HTTP)
+    check("attachment; filename==?ISO-8859-1?Q?foo-=E4.html?=", "foo-\u00e4.html");
+    check("attachment; filename=\"=?ISO-8859-1?Q?foo-=E4.html?=\"", "foo-\u00e4.html");
+    // format sent by GMail as of 2012-07-23 (5987 overrides 2047)
+    check("attachment; filename=\"=?ISO-8859-1?Q?foo-=E4.html?=\"; filename*=UTF-8''5987", "5987");
+    // Bug 651185: double quotes around 2231/5987 encoded param
+    // Change reverted to backwards compat issues with various web services,
+    // such as OWA (Bug 703015), plus similar problems in Thunderbird. If this
+    // is tried again in the future, email probably needs to be special-cased.
+    // sanity check
+    check("attachment; filename*=utf-8''%41", "A");
+    // the actual bug
+    check(`attachment; filename*=${DQUOTE}utf-8''%41${DQUOTE}`, "A");
+    // Bug 670333: Content-Disposition parser does not require presence of "="
+    // in params
+    // sanity check
+    check("attachment; filename*=UTF-8''foo-%41.html", "foo-A.html");
+    // the actual bug
+    check("attachment; filename *=UTF-8''foo-%41.html", "");
+    // the actual bug, without 2231/5987 encoding
+    check("attachment; filename X", "");
+    // sanity check with WS on both sides
+    check("attachment; filename = foo-A.html", "foo-A.html");
+    // Bug 685192: in RFC2231/5987 encoding, a missing charset field should be
+    // treated as error
+    // the actual bug
+    check("attachment; filename*=''foo", "foo");
+    // sanity check
+    check("attachment; filename*=a''foo", "foo");
+    // Bug 692574: RFC2231/5987 decoding should not tolerate missing single
+    // quotes
+    // one missing
+    check("attachment; filename*=UTF-8'foo-%41.html", "foo-A.html");
+    // both missing
+    check("attachment; filename*=foo-%41.html", "foo-A.html");
+    // make sure fallback works
+    check("attachment; filename*=UTF-8'foo-%41.html; filename=bar.html", "foo-A.html");
+    // Bug 693806: RFC2231/5987 encoding: charset information should be treated
+    // as authoritative
+    // UTF-8 labeled ISO-8859-1
+    check("attachment; filename*=ISO-8859-1''%c3%a4", "\u00c3\u00a4");
+    // UTF-8 labeled ISO-8859-1, but with octets not allowed in ISO-8859-1
+    // accepts x82, understands it as Win1252, maps it to Unicode \u20a1
+    check("attachment; filename*=ISO-8859-1''%e2%82%ac", "\u00e2\u201a\u00ac");
+    // defective UTF-8
+    nocheck("attachment; filename*=UTF-8''A%e4B", "");
+    // defective UTF-8, with fallback
+    nocheck("attachment; filename*=UTF-8''A%e4B; filename=fallback", "fallback");
+    // defective UTF-8 (continuations), with fallback
+    nocheck("attachment; filename*0*=UTF-8''A%e4B; filename=fallback", "fallback");
+    // check that charsets aren't mixed up
+    check("attachment; filename*0*=ISO-8859-15''euro-sign%3d%a4; filename*=ISO-8859-1''currency-sign%3d%a4", "currency-sign=\u00a4");
+    // same as above, except reversed
+    check("attachment; filename*=ISO-8859-1''currency-sign%3d%a4; filename*0*=ISO-8859-15''euro-sign%3d%a4", "currency-sign=\u00a4");
+    // Bug 704989: add workaround for broken Outlook Web App (OWA)
+    // attachment handling
+    check("attachment; filename*=\"a%20b\"", "a b");
+    // Bug 717121: crash nsMIMEHeaderParamImpl::DoParameterInternal
+    check("attachment; filename=\"", "");
+    // We used to read past string if last param w/o = and ;
+    // Note: was only detected on windows PGO builds
+    check("attachment; filename=foo; trouble", "foo");
+    // Same, followed by space, hits another case
+    check("attachment; filename=foo; trouble ", "foo");
+    check("attachment", "");
+    // Bug 730574: quoted-string in RFC2231-continuations not handled
+    check("attachment; filename=basic; filename*0=\"foo\"; filename*1=\"\\b\\a\\r.html\"", "foobar.html");
+    // unmatched escape char
+    check("attachment; filename=basic; filename*0=\"foo\"; filename*1=\"\\b\\a\\", "fooba\\");
+    // Bug 732369: Content-Disposition parser does not require presence of ";" between params
+    // optimally, this would not even return the disposition type "attachment"
+    check("attachment; extension=bla filename=foo", "");
+    check("attachment; filename=foo extension=bla", "foo");
+    check("attachment filename=foo", "");
+    // Bug 777687: handling of broken %escapes
+    nocheck("attachment; filename*=UTF-8''f%oo; filename=bar", "bar");
+    nocheck("attachment; filename*=UTF-8''foo%; filename=bar", "bar");
+    // Bug 783502 - xpcshell test netwerk/test/unit/test_MIME_params.js fails on AddressSanitizer
+    check("attachment; filename=\"\\b\\a\\", "ba\\");
+  });
+
+  it("parse extra", function() {
+    // Extra tests, not covered by above tests.
+    check("inline; FILENAME=file.txt", "file.txt");
+    check("INLINE; FILENAME= \"an example.html\"", "an example.html"); // RFC 6266, section 5.
+    check("inline; filename= \"tl;dr.txt\"", "tl;dr.txt");
+    check("INLINE; FILENAME*= \"an example.html\"", "an example.html");
+    check("inline; filename*= \"tl;dr.txt\"", "tl;dr.txt");
+    check("inline; filename*0=\"tl;dr and \"; filename*1=more.txt", "tl;dr and more.txt");
+  });
+
+  it("parse issue 26", function() {
+    // https://github.com/Rob--W/open-in-browser/issues/26
+    check("attachment; filename=\xe5\x9c\x8b.pdf", "\u570b.pdf");
+  });
+
+  it("parse issue 35", function() {
+    // https://github.com/Rob--W/open-in-browser/issues/35
+    check("attachment; filename=okre\x9clenia.rtf", "okreœlenia.rtf");
+  });
+});

tests/test_mime.js

@@ -0,0 +1,30 @@
+"use strict";
+// License: CC0 1.0
+
+// eslint-disable-next-line @typescript-eslint/no-var-requires
+const {MimeDB} = require("../lib/mime");
+
+describe("MIME", function() {
+  it("general", function() {
+    expect(MimeDB.getMime("image/jpeg").major).to.equal("image");
+    expect(MimeDB.getMime("image/jpeg").minor).to.equal("jpeg");
+    expect(MimeDB.getMime("iMage/jPeg").major).to.equal("image");
+    expect(MimeDB.getMime("imAge/jpEg").minor).to.equal("jpeg");
+  });
+
+  it("exts", function() {
+    expect(MimeDB.getMime("image/jpeg").primary).to.equal("jpg");
+    expect(MimeDB.getMime("image/jpeg").primary).to.equal(
+      MimeDB.getPrimary("image/jpeg"));
+    expect(MimeDB.getMime("iMage/jPeg").primary).to.equal("jpg");
+    expect(MimeDB.getMime("imAge/jpEg").primary).to.equal(
+      MimeDB.getPrimary("image/jpeg"));
+    expect(Array.from(MimeDB.getMime("imAge/jpEg").extensions)).to.deep.equal(
+      ["jpg", "jpeg", "jpe", "jfif"]);
+  });
+
+  it("application/octet-stream should not yield results", function() {
+    expect(MimeDB.getPrimary("application/octet-stream")).to.equal("");
+    expect(MimeDB.getMime("application/octet-Stream")).to.be.undefined;
+  });
+});

tests/test_renamer.js

@@ -19,7 +19,7 @@ const OPTS = {
   state: DownloadState.QUEUED,
   batch: 42,
   idx: 23,
-  mask: "*name*.*ext",
+  mask: "*name*.*ext*",
   description: "desc / ript.ion .",
   title: " *** TITLE *** ",
 };
@@ -57,6 +57,49 @@ describe("Renamer", function() {
       expect(dest.path).to.equal("");
     });
 
+    it("*name*.*ext* (mime override)", function() {
+      const {dest} = new BaseDownload(
+        Object.assign({}, OPTS, {
+          mask: "*name* *batch*.*ext*",
+          mime: "image/jpeg"
+        }));
+      expect(dest.full).to.equal("filenäme 042.jpg");
+      expect(dest.name).to.equal("filenäme 042.jpg");
+      expect(dest.base).to.equal("filenäme 042");
+      expect(dest.ext).to.equal("jpg");
+      expect(dest.path).to.equal("");
+    });
+
+    it("*name*.*ext* (mime no override)", function() {
+      const {dest} = new BaseDownload(
+        Object.assign({}, OPTS, {
+          mask: "*name* *batch*.*ext*",
+          mime: "image/jpeg",
+          url: "https://www.example.co.uk/filen%C3%A4me.JPe",
+          usable: "https://www.example.co.uk/filenäme.JPe",
+        }));
+      expect(dest.full).to.equal("filenäme 042.JPe");
+      expect(dest.name).to.equal("filenäme 042.JPe");
+      expect(dest.base).to.equal("filenäme 042");
+      expect(dest.ext).to.equal("JPe");
+      expect(dest.path).to.equal("");
+    });
+
+    it("*name*.*ext* (mime override; missing ext)", function() {
+      const {dest} = new BaseDownload(
+        Object.assign({}, OPTS, {
+          mask: "*name* *batch*.*ext*",
+          mime: "application/json",
+          url: "https://www.example.co.uk/filen%C3%A4me",
+          usable: "https://www.example.co.uk/filenäme",
+        }));
+      expect(dest.full).to.equal("filenäme 042.json");
+      expect(dest.name).to.equal("filenäme 042.json");
+      expect(dest.base).to.equal("filenäme 042");
+      expect(dest.ext).to.equal("json");
+      expect(dest.path).to.equal("");
+    });
+
     it("*text*", function() {
       const dest = makeOne("*text*");
       expect(dest.full).to.equal("desc/ript.ion");

tests/test_urld.js

@@ -0,0 +1,43 @@
+/* eslint-env node */
+/* eslint-disable @typescript-eslint/no-var-requires */
+"use strict";
+// License: CC0 1.0
+
+require("../lib/util");
+
+describe("URLd", function() {
+  it("basic domain", function() {
+    let u = new URL("https://www.google.de");
+    expect(u.domain).to.equal("google.de");
+    u = new URL("https://www.google.de:8443");
+    expect(u.domain).to.equal("google.de");
+  });
+
+  it("plain basic domain", function() {
+    const u = new URL("https://google.de");
+    expect(u.domain).to.equal("google.de");
+  });
+
+  it("special domain", function() {
+    let u = new URL("https://www.google.co.uk");
+    expect(u.domain).to.equal("google.co.uk");
+    u = new URL("https://google.co.uk");
+    expect(u.domain).to.equal("google.co.uk");
+    u = new URL("https://www.google.co.uk:8443");
+    expect(u.domain).to.equal("google.co.uk");
+  });
+
+  it("ipv4", function() {
+    let u = new URL("https://127.0.0.1:8443");
+    expect(u.domain).to.equal("127.0.0.1");
+    u = new URL("https://0.0.0.0:8443");
+    expect(u.domain).to.equal("0.0.0.0");
+  });
+
+  it("ipv6", function() {
+    let u = new URL("https://[::1]:8443");
+    expect(u.domain).to.equal("[::1]");
+    u = new URL("https://[2a00:1450:4005:800::2003]:8443");
+    expect(u.domain).to.equal("[2a00:1450:4005:800::2003]");
+  });
+});

tsconfig.json

@@ -12,6 +12,8 @@
     "noImplicitReturns": true,
     "noUnusedLocals": true,
     "resolveJsonModule": true,
+    "esModuleInterop": true,
+    "importHelpers": true,
     "sourceMap": true
   }
 }

uikit/css/modal.css

@@ -33,7 +33,7 @@
 .modal-footer {
   display: flex;
   flex-wrap: nowrap;
-  justify-content: right;
+  justify-content: flex-end;
   background: rgba(30, 30, 30, 0.2);
   margin-top: 2em;
   border-top: 1px solid rgba(30, 30, 30, 0.5);

uikit/lib/contextmenu.ts

@@ -108,8 +108,13 @@ export class MenuItem extends MenuItemBase {
     super(owner, id, text, options);
     this.disabled = options.disabled === "true";
     this.elem.setAttribute("aria-role", "menuitem");
-    this.elem.addEventListener(
-      "click", () => this.owner.emit("clicked", this.id, this.autoHide));
+    this.clicked = this.clicked.bind(this);
+    this.elem.addEventListener("click", this.clicked);
+    this.elem.addEventListener("contextmenu", this.clicked);
+  }
+
+  clicked() {
+    this.owner.emit("clicked", this.id, this.autoHide);
   }
 
   get disabled() {