lordwelch/comic-hasher
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
840e97cff0
comic-hasher/go.mod

62 lines
2.4 KiB
Modula-2
Raw Normal View History

more stuff
2024-08-04 18:12:00 -07:00
module gitea.narnian.us/lordwelch/comic-hasher
Initial Commit
2024-05-01 18:09:02 -07:00
Optimize memory usage Add a basic map storage that does manual searches to conserve memory Change saved hashes format to allow multiple hashes for a given ID Add a vptree storage Maps in Go take up a huge amount of space changing IDList to []ID took memory from over 1GB down to 200MB (note this was on aarch64 MacOS which for some reason uses less memory than aarch64 Linux). Exhaustive searches using slices took about 30 ms search now takes 50-60 ms as it takes longer to iterate a map. Partial hashes will speed up searches to 8 ms at the cost of 700MB initial memory usage and 400MB idle (though this is on MacOS, which for some reason uses less memory that aarch64 Linux so probably more like 900MB initial -> 600 MB idle on an RPI running Linux)
2024-09-07 14:51:18 -07:00
go 1.23.0
Initial Commit
2024-05-01 18:09:02 -07:00
require (
Finish POC Make a pip installable package for quick-tag Save and load hashes from msgpack/json Embed hashes of all thumb_urls from ComicVine Allow associating new IDs to old IDs Support multiple domains Support hashing local covers Support simple list of ids and distance and results including the hash
2024-08-11 20:46:41 -07:00
gitea.narnian.us/lordwelch/goimagehash v0.0.0-20240812025715-33ff96e45f00
stuff
2024-07-31 11:35:17 -07:00
github.com/fmartingr/go-comicinfo/v2 v2.0.2
Make runtime hash storage modular
2024-09-01 18:13:47 -07:00
github.com/kr/pretty v0.1.0
Add sqlite implementation
2024-09-01 18:05:58 -07:00
github.com/mattn/go-sqlite3 v1.14.22
stuff
2024-07-31 11:35:17 -07:00
github.com/mholt/archiver/v4 v4.0.0-alpha.8
Optimize memory usage Add a basic map storage that does manual searches to conserve memory Change saved hashes format to allow multiple hashes for a given ID Add a vptree storage Maps in Go take up a huge amount of space changing IDList to []ID took memory from over 1GB down to 200MB (note this was on aarch64 MacOS which for some reason uses less memory than aarch64 Linux). Exhaustive searches using slices took about 30 ms search now takes 50-60 ms as it takes longer to iterate a map. Partial hashes will speed up searches to 8 ms at the cost of 700MB initial memory usage and 400MB idle (though this is on MacOS, which for some reason uses less memory that aarch64 Linux so probably more like 900MB initial -> 600 MB idle on an RPI running Linux)
2024-09-07 14:51:18 -07:00
github.com/ncruces/go-sqlite3 v0.18.1
Finish POC Make a pip installable package for quick-tag Save and load hashes from msgpack/json Embed hashes of all thumb_urls from ComicVine Allow associating new IDs to old IDs Support multiple domains Support hashing local covers Support simple list of ids and distance and results including the hash
2024-08-11 20:46:41 -07:00
golang.org/x/image v0.19.0
golang.org/x/text v0.17.0
Optimize memory usage Add a basic map storage that does manual searches to conserve memory Change saved hashes format to allow multiple hashes for a given ID Add a vptree storage Maps in Go take up a huge amount of space changing IDList to []ID took memory from over 1GB down to 200MB (note this was on aarch64 MacOS which for some reason uses less memory than aarch64 Linux). Exhaustive searches using slices took about 30 ms search now takes 50-60 ms as it takes longer to iterate a map. Partial hashes will speed up searches to 8 ms at the cost of 700MB initial memory usage and 400MB idle (though this is on MacOS, which for some reason uses less memory that aarch64 Linux so probably more like 900MB initial -> 600 MB idle on an RPI running Linux)
2024-09-07 14:51:18 -07:00
gonum.org/v1/gonum v0.15.1
Add sqlite implementation
2024-09-01 18:05:58 -07:00
modernc.org/sqlite v1.32.0
Finish POC Make a pip installable package for quick-tag Save and load hashes from msgpack/json Embed hashes of all thumb_urls from ComicVine Allow associating new IDs to old IDs Support multiple domains Support hashing local covers Support simple list of ids and distance and results including the hash
2024-08-11 20:46:41 -07:00
)
require (
github.com/vmihailenco/msgpack/v5 v5.4.1
github.com/vmihailenco/tagparser/v2 v2.0.0 // indirect
Initial Commit
2024-05-01 18:09:02 -07:00
)
require (
stuff
2024-07-31 11:35:17 -07:00
github.com/andybalholm/brotli v1.0.4 // indirect
github.com/bodgit/plumbing v1.2.0 // indirect
github.com/bodgit/sevenzip v1.3.0 // indirect
github.com/bodgit/windows v1.0.0 // indirect
github.com/connesc/cipherio v0.2.1 // indirect
Make runtime hash storage modular
2024-09-01 18:13:47 -07:00
github.com/disintegration/imaging v1.6.3-0.20201218193011-d40f48ce0f09 // indirect
stuff
2024-07-31 11:35:17 -07:00
github.com/dsnet/compress v0.0.1 // indirect
Add sqlite implementation
2024-09-01 18:05:58 -07:00
github.com/dustin/go-humanize v1.0.1 // indirect
more stuff
2024-08-04 18:12:00 -07:00
github.com/golang/mock v1.6.0 // indirect
stuff
2024-07-31 11:35:17 -07:00
github.com/golang/snappy v0.0.4 // indirect
Add sqlite implementation
2024-09-01 18:05:58 -07:00
github.com/google/uuid v1.6.0 // indirect
stuff
2024-07-31 11:35:17 -07:00
github.com/hashicorp/errwrap v1.0.0 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
Add sqlite implementation
2024-09-01 18:05:58 -07:00
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
stuff
2024-07-31 11:35:17 -07:00
github.com/klauspost/compress v1.15.9 // indirect
github.com/klauspost/pgzip v1.2.5 // indirect
Make runtime hash storage modular
2024-09-01 18:13:47 -07:00
github.com/kr/text v0.1.0 // indirect
Add sqlite implementation
2024-09-01 18:05:58 -07:00
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/ncruces/go-strftime v0.1.9 // indirect
Optimize memory usage Add a basic map storage that does manual searches to conserve memory Change saved hashes format to allow multiple hashes for a given ID Add a vptree storage Maps in Go take up a huge amount of space changing IDList to []ID took memory from over 1GB down to 200MB (note this was on aarch64 MacOS which for some reason uses less memory than aarch64 Linux). Exhaustive searches using slices took about 30 ms search now takes 50-60 ms as it takes longer to iterate a map. Partial hashes will speed up searches to 8 ms at the cost of 700MB initial memory usage and 400MB idle (though this is on MacOS, which for some reason uses less memory that aarch64 Linux so probably more like 900MB initial -> 600 MB idle on an RPI running Linux)
2024-09-07 14:51:18 -07:00
github.com/ncruces/julianday v1.0.0 // indirect
stuff
2024-07-31 11:35:17 -07:00
github.com/nwaples/rardecode/v2 v2.0.0-beta.2 // indirect
github.com/pierrec/lz4/v4 v4.1.15 // indirect
Add sqlite implementation
2024-09-01 18:05:58 -07:00
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
Optimize memory usage Add a basic map storage that does manual searches to conserve memory Change saved hashes format to allow multiple hashes for a given ID Add a vptree storage Maps in Go take up a huge amount of space changing IDList to []ID took memory from over 1GB down to 200MB (note this was on aarch64 MacOS which for some reason uses less memory than aarch64 Linux). Exhaustive searches using slices took about 30 ms search now takes 50-60 ms as it takes longer to iterate a map. Partial hashes will speed up searches to 8 ms at the cost of 700MB initial memory usage and 400MB idle (though this is on MacOS, which for some reason uses less memory that aarch64 Linux so probably more like 900MB initial -> 600 MB idle on an RPI running Linux)
2024-09-07 14:51:18 -07:00
github.com/tetratelabs/wazero v1.8.0 // indirect
stuff
2024-07-31 11:35:17 -07:00
github.com/therootcompany/xz v1.0.1 // indirect
github.com/ulikunitz/xz v0.5.10 // indirect
go4.org v0.0.0-20200411211856-f5505b9728dd // indirect
Finish POC Make a pip installable package for quick-tag Save and load hashes from msgpack/json Embed hashes of all thumb_urls from ComicVine Allow associating new IDs to old IDs Support multiple domains Support hashing local covers Support simple list of ids and distance and results including the hash
2024-08-11 20:46:41 -07:00
golang.org/x/exp v0.0.0-20240808152545-0cdaa3abc0fa // indirect
Optimize memory usage Add a basic map storage that does manual searches to conserve memory Change saved hashes format to allow multiple hashes for a given ID Add a vptree storage Maps in Go take up a huge amount of space changing IDList to []ID took memory from over 1GB down to 200MB (note this was on aarch64 MacOS which for some reason uses less memory than aarch64 Linux). Exhaustive searches using slices took about 30 ms search now takes 50-60 ms as it takes longer to iterate a map. Partial hashes will speed up searches to 8 ms at the cost of 700MB initial memory usage and 400MB idle (though this is on MacOS, which for some reason uses less memory that aarch64 Linux so probably more like 900MB initial -> 600 MB idle on an RPI running Linux)
2024-09-07 14:51:18 -07:00
golang.org/x/sys v0.24.0 // indirect
Add sqlite implementation
2024-09-01 18:05:58 -07:00
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 // indirect
modernc.org/libc v1.55.3 // indirect
modernc.org/mathutil v1.6.0 // indirect
modernc.org/memory v1.8.0 // indirect
modernc.org/strutil v1.2.0 // indirect
modernc.org/token v1.1.0 // indirect
Initial Commit
2024-05-01 18:09:02 -07:00
)
Finish POC Make a pip installable package for quick-tag Save and load hashes from msgpack/json Embed hashes of all thumb_urls from ComicVine Allow associating new IDs to old IDs Support multiple domains Support hashing local covers Support simple list of ids and distance and results including the hash
2024-08-11 20:46:41 -07:00
replace golang.org/x/text v0.17.0 => github.com/lordwelch/text v0.0.0-20240505231825-4893f344170f
Reference in New Issue Copy Permalink