Mercurial > touhou
annotate pytouhou/formats/pbg3.py @ 778:816e1f01d650
Partially replace the Loader with a Rust one
| author | Link Mauve <linkmauve@linkmauve.fr> |
|---|---|
| date | Sat, 08 Nov 2025 18:26:01 +0100 |
| parents | d18c0bf11138 |
| children |
| rev | line source |
|---|---|
|
52
ab826bc29aa2
Add some documentation, GPLv3 headers, README and COPYING file.
Thibaut Girka <thib@sitedethib.com>
parents:
0
diff
changeset
|
1 # -*- encoding: utf-8 -*- |
|
ab826bc29aa2
Add some documentation, GPLv3 headers, README and COPYING file.
Thibaut Girka <thib@sitedethib.com>
parents:
0
diff
changeset
|
2 ## |
|
ab826bc29aa2
Add some documentation, GPLv3 headers, README and COPYING file.
Thibaut Girka <thib@sitedethib.com>
parents:
0
diff
changeset
|
3 ## Copyright (C) 2011 Thibaut Girka <thib@sitedethib.com> |
|
ab826bc29aa2
Add some documentation, GPLv3 headers, README and COPYING file.
Thibaut Girka <thib@sitedethib.com>
parents:
0
diff
changeset
|
4 ## |
|
ab826bc29aa2
Add some documentation, GPLv3 headers, README and COPYING file.
Thibaut Girka <thib@sitedethib.com>
parents:
0
diff
changeset
|
5 ## This program is free software; you can redistribute it and/or modify |
|
ab826bc29aa2
Add some documentation, GPLv3 headers, README and COPYING file.
Thibaut Girka <thib@sitedethib.com>
parents:
0
diff
changeset
|
6 ## it under the terms of the GNU General Public License as published |
|
ab826bc29aa2
Add some documentation, GPLv3 headers, README and COPYING file.
Thibaut Girka <thib@sitedethib.com>
parents:
0
diff
changeset
|
7 ## by the Free Software Foundation; version 3 only. |
|
ab826bc29aa2
Add some documentation, GPLv3 headers, README and COPYING file.
Thibaut Girka <thib@sitedethib.com>
parents:
0
diff
changeset
|
8 ## |
|
ab826bc29aa2
Add some documentation, GPLv3 headers, README and COPYING file.
Thibaut Girka <thib@sitedethib.com>
parents:
0
diff
changeset
|
9 ## This program is distributed in the hope that it will be useful, |
|
ab826bc29aa2
Add some documentation, GPLv3 headers, README and COPYING file.
Thibaut Girka <thib@sitedethib.com>
parents:
0
diff
changeset
|
10 ## but WITHOUT ANY WARRANTY; without even the implied warranty of |
|
ab826bc29aa2
Add some documentation, GPLv3 headers, README and COPYING file.
Thibaut Girka <thib@sitedethib.com>
parents:
0
diff
changeset
|
11 ## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
|
ab826bc29aa2
Add some documentation, GPLv3 headers, README and COPYING file.
Thibaut Girka <thib@sitedethib.com>
parents:
0
diff
changeset
|
12 ## GNU General Public License for more details. |
|
ab826bc29aa2
Add some documentation, GPLv3 headers, README and COPYING file.
Thibaut Girka <thib@sitedethib.com>
parents:
0
diff
changeset
|
13 ## |
|
ab826bc29aa2
Add some documentation, GPLv3 headers, README and COPYING file.
Thibaut Girka <thib@sitedethib.com>
parents:
0
diff
changeset
|
14 |
|
204
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
15 """PBG3 archive files handling. |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
16 |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
17 This module provides classes for handling the PBG3 file format. |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
18 The PBG3 format is the archive format used by Touhou: EoSD. |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
19 |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
20 PBG3 files are merely a bitstream composed of a header, |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
21 a file table, and LZSS-compressed files. |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
22 """ |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
23 |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
24 from collections import namedtuple |
|
536
6b76c9ba3975
Make archives return files by default, instead of bytes.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
490
diff
changeset
|
25 from io import BytesIO |
|
204
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
26 |
| 58 | 27 from pytouhou.utils.bitstream import BitStream |
|
490
1b532e7dd521
Decrease PBG3 loading time by improving lzss and bitstream integration.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
377
diff
changeset
|
28 from pytouhou.utils import lzss |
| 58 | 29 |
| 30 from pytouhou.utils.helpers import get_logger | |
| 31 | |
|
377
70e2ed71b09c
Add meaningful exceptions in format parsing.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
252
diff
changeset
|
32 from pytouhou.formats import WrongFormatError |
|
70e2ed71b09c
Add meaningful exceptions in format parsing.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
252
diff
changeset
|
33 |
| 58 | 34 logger = get_logger(__name__) |
| 35 | |
| 36 | |
| 0 | 37 class PBG3BitStream(BitStream): |
|
204
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
38 """Helper class to handle strings and integers in PBG3 bitstreams.""" |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
39 |
| 0 | 40 def read_int(self): |
|
204
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
41 """Read an integer from the bitstream. |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
42 |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
43 Integers have variable sizes. They begin with a two-bit value indicating |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
44 the number of (non-aligned) bytes to read. |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
45 """ |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
46 |
| 0 | 47 size = self.read(2) |
| 48 return self.read((size + 1) * 8) | |
| 49 | |
| 50 | |
| 51 def read_string(self, maxsize): | |
|
204
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
52 """Read a string from the bitstream. |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
53 |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
54 Strings are stored as standard NULL-termianted sequences of bytes. |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
55 The only catch is that they are not byte-aligned. |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
56 """ |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
57 |
| 0 | 58 string = [] |
| 59 for i in range(maxsize): | |
| 60 byte = self.read(8) | |
| 61 if byte == 0: | |
| 62 break | |
| 63 string.append(byte) | |
| 64 return ''.join(chr(byte) for byte in string) | |
| 65 | |
| 66 | |
| 67 | |
|
204
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
68 PBG3Entry = namedtuple('PBG3Entry', 'unknown1 unknown2 checksum offset size') |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
69 |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
70 |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
71 |
|
615
d1f0bb0b7a17
Don’t inherit explicitely from object, we are not on Python 2.7 anymore. :)
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
590
diff
changeset
|
72 class PBG3: |
|
204
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
73 """Handle PBG3 archive files. |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
74 |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
75 PBG3 is a file archive format used in Touhou 6: EoSD. |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
76 This class provides a representation of such files, as well as functions to |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
77 read and extract files from a PBG3 archive. |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
78 |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
79 Instance variables: |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
80 entries -- list of PBG3Entry objects describing files present in the archive |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
81 bitstream -- PBG3BitStream object |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
82 """ |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
83 |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
84 def __init__(self, entries=None, bitstream=None): |
|
252
b5c7369abd7c
Improve data reading perfs
Thibaut Girka <thib@sitedethib.com>
parents:
204
diff
changeset
|
85 self.entries = entries or {} |
| 0 | 86 self.bitstream = bitstream #TODO |
| 87 | |
| 88 | |
| 97 | 89 def __enter__(self): |
| 90 return self | |
| 91 | |
| 92 | |
| 93 def __exit__(self, type, value, traceback): | |
| 94 return self.bitstream.__exit__(type, value, traceback) | |
| 95 | |
| 96 | |
| 0 | 97 @classmethod |
| 98 def read(cls, file): | |
|
204
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
99 """Read a PBG3 file. |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
100 |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
101 Raise an exception if the file is invalid. |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
102 Return a PBG3 instance otherwise. |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
103 """ |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
104 |
| 0 | 105 magic = file.read(4) |
| 106 if magic != b'PBG3': | |
|
377
70e2ed71b09c
Add meaningful exceptions in format parsing.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
252
diff
changeset
|
107 raise WrongFormatError(magic) |
| 0 | 108 |
| 109 bitstream = PBG3BitStream(file) | |
| 110 entries = {} | |
| 111 | |
| 112 nb_entries = bitstream.read_int() | |
| 113 offset = bitstream.read_int() | |
| 114 bitstream.seek(offset) | |
| 115 for i in range(nb_entries): | |
| 116 unknown1 = bitstream.read_int() | |
| 117 unknown2 = bitstream.read_int() | |
| 118 checksum = bitstream.read_int() # Checksum of *compressed data* | |
| 119 offset = bitstream.read_int() | |
| 120 size = bitstream.read_int() | |
|
590
e15672733c93
Switch to Python 3.x instead of 2.7.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
536
diff
changeset
|
121 name = bitstream.read_string(255) |
|
204
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
122 entries[name] = PBG3Entry(unknown1, unknown2, checksum, offset, size) |
| 0 | 123 |
| 124 return PBG3(entries, bitstream) | |
| 125 | |
| 126 | |
| 127 def list_files(self): | |
|
204
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
128 """List files present in the archive.""" |
| 0 | 129 return self.entries.keys() |
| 130 | |
| 131 | |
|
536
6b76c9ba3975
Make archives return files by default, instead of bytes.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
490
diff
changeset
|
132 def get_file(self, filename, check=False): |
|
204
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
133 """Extract a given file. |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
134 |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
135 If “filename” is in the archive, extract it and return its contents. |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
136 Otherwise, raise an exception. |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
137 |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
138 By default, the checksum of the file won't be verified, |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
139 you can however force the verification using the “check” argument. |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
140 """ |
|
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
141 |
| 0 | 142 unkwn1, unkwn2, checksum, offset, size = self.entries[filename] |
| 143 self.bitstream.seek(offset) | |
| 144 data = lzss.decompress(self.bitstream, size) | |
| 145 if check: | |
|
204
88361534c77e
Add some documentation (argh, so much left to document!)
Thibaut Girka <thib@sitedethib.com>
parents:
97
diff
changeset
|
146 # Verify the checksum |
| 0 | 147 compressed_size = self.bitstream.io.tell() - offset |
| 148 self.bitstream.seek(offset) | |
| 149 value = 0 | |
| 150 for c in self.bitstream.io.read(compressed_size): | |
|
590
e15672733c93
Switch to Python 3.x instead of 2.7.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
536
diff
changeset
|
151 value += c |
| 0 | 152 value &= 0xFFFFFFFF |
| 153 if value != checksum: | |
|
764
d18c0bf11138
Python: Use logger.warning() as logger.warn() is deprecated
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
615
diff
changeset
|
154 logger.warning('corrupted data!') |
|
536
6b76c9ba3975
Make archives return files by default, instead of bytes.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
490
diff
changeset
|
155 return BytesIO(data) |
