Mercurial > danboorufs

annotate danboorufs.py @ 6:2c81cc41de2d draft

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Allow only a base directory for the tags, which will be recursively walked to find the actual tags files.
author Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
date Sun, 02 Dec 2012 19:15:17 +0100
parents a422e75bf464
children 09945ce42e28
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3
880904f1071f Add python3 support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 2
diff changeset
1 #!/usr/bin/env python
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
2 # -*- encoding: utf-8 -*-
2
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
3 #
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
4 #
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
5 # Copyright © 2012 Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
6 #
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
7 # Permission is hereby granted, free of charge, to any person obtaining a copy
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
8 # of this software and associated documentation files (the "Software"), to deal
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
9 # in the Software without restriction, including without limitation the rights
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
10 # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
11 # copies of the Software, and to permit persons to whom the Software is
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
12 # furnished to do so, subject to the following conditions:
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
13 #
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
14 # The above copyright notice and this permission notice shall be included in
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
15 # all copies or substantial portions of the Software.
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
16 #
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
17 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
18 # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
19 # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
20 # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
21 # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
22 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
23 # SOFTWARE.
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
24
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
25
3
880904f1071f Add python3 support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 2
diff changeset
26 from __future__ import with_statement, unicode_literals
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
27
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
28 from errno import ENOENT, ENOTDIR
2
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
29 from sys import argv
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
30 from threading import Lock
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
31 from time import time
3
880904f1071f Add python3 support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 2
diff changeset
32 from functools import reduce
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
33
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
34 import os
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
35
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
36 from fuse import FUSE, FuseOSError, Operations, LoggingMixIn
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
37
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
38
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
39 class Danbooru(LoggingMixIn, Operations):
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
40 '''
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
41 Represent a list of images as a filesystem tree, with nice tag filtering.
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
42 '''
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
43
4
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
44 def __init__(self, tagfiles, root, use_symlinks):
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
45 '''
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
46 Takes a list of files containing the tags. They have to be named as the
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
47 image, with ".tags" at the end.
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
48 '''
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
49 self.paths = {}
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
50 self.files = {}
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
51 self.tags = {}
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
52 self.cache = {}
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
53
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
54 start = time()
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
55
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
56 for name in tagfiles:
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
57 filename = name.replace('.tags', '')
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
58 basename = os.path.basename(filename)
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
59 self.paths[basename] = filename
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
60 tags = []
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
61 self.files[basename] = tags
6
2c81cc41de2d Allow only a base directory for the tags, which will be recursively walked to find the actual tags files.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 5
diff changeset
62 with open(name, 'rb') as tagfile:
2
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
63 for line in tagfile:
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
64 for tag in line.split():
3
880904f1071f Add python3 support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 2
diff changeset
65 try:
880904f1071f Add python3 support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 2
diff changeset
66 tag = tag.decode('UTF-8')
6
2c81cc41de2d Allow only a base directory for the tags, which will be recursively walked to find the actual tags files.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 5
diff changeset
67 except UnicodeDecodeError:
2c81cc41de2d Allow only a base directory for the tags, which will be recursively walked to find the actual tags files.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 5
diff changeset
68 continue
3
880904f1071f Add python3 support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 2
diff changeset
69 except AttributeError:
880904f1071f Add python3 support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 2
diff changeset
70 pass
880904f1071f Add python3 support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 2
diff changeset
71 tag = tag.replace('/', '�') #XXX
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
72 tags.append(tag)
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
73 self.tags.setdefault(tag, []).append(basename)
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
74
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
75 print('[%d] Index done.' % (time() - start))
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
76
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
77 self.root = root
4
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
78 self.use_symlinks = use_symlinks
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
79 self.rwlock = Lock()
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
80
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
81 def _split_path(self, path):
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
82 if path == '/':
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
83 return (None, None)
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
84
1
63ccd8b0d615 Add tag exclusion support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 0
diff changeset
85 real_path = path[1:].split('/')
63ccd8b0d615 Add tag exclusion support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 0
diff changeset
86
63ccd8b0d615 Add tag exclusion support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 0
diff changeset
87 # Remove the leading - of tag exclusion.
63ccd8b0d615 Add tag exclusion support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 0
diff changeset
88 path = [tag[1:] if tag[0] == '-' else tag for tag in real_path]
63ccd8b0d615 Add tag exclusion support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 0
diff changeset
89
2
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
90 for tag in path[:-1]:
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
91 if tag not in self.tags:
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
92 raise FuseOSError(ENOENT)
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
93
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
94 if path[-1] in self.tags:
1
63ccd8b0d615 Add tag exclusion support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 0
diff changeset
95 return (real_path, None)
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
96
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
97 if path[-1] not in self.paths:
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
98 raise FuseOSError(ENOENT)
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
99
1
63ccd8b0d615 Add tag exclusion support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 0
diff changeset
100 return (real_path[:-1], self.paths[real_path[-1]])
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
101
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
102 def access(self, path, mode):
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
103 self._split_path(path)
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
104
2
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
105 def getattr(self, path, file_handle=None):
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
106 _, filename = self._split_path(path)
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
107 path = filename if filename else self.root
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
108 stat = os.lstat(path)
4
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
109 stat = dict((key, getattr(stat, key)) for key in ('st_atime',
2
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
110 'st_ctime', 'st_gid', 'st_mode', 'st_mtime',
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
111 'st_nlink', 'st_size', 'st_uid'))
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
112
4
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
113 if self.use_symlinks:
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
114 # Those modes are respectively for a symlink and a directory.
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
115 stat['st_mode'] = 0o120700 if filename else 0o40700
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
116 if filename:
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
117 stat['st_size'] = len(filename)
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
118
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
119 return stat
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
120
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
121 getxattr = None
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
122 listxattr = None
2
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
123
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
124 def open(self, path, flags):
2
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
125 _, filename = self._split_path(path)
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
126 return os.open(filename, flags)
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
127
2
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
128 def read(self, path, size, offset, file_handle):
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
129 with self.rwlock:
2
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
130 os.lseek(file_handle, offset, 0)
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
131 return os.read(file_handle, size)
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
132
2
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
133 def readdir(self, path, file_handle):
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
134 if path == '/':
3
880904f1071f Add python3 support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 2
diff changeset
135 return (['.', '..'] + list(self.tags.keys())
880904f1071f Add python3 support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 2
diff changeset
136 + list(self.files.keys()))
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
137
2
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
138 tags, filename = self._split_path(path)
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
139 if filename:
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
140 return FuseOSError(ENOTDIR)
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
141
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
142 tags = set(tags)
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
143
2
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
144 key = ' '.join(tags)
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
145 if key in self.cache:
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
146 return ['.', '..'] + self.cache[key]
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
147
1
63ccd8b0d615 Add tag exclusion support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 0
diff changeset
148 inclusion_tags = set(tag for tag in tags if tag[0] != '-')
63ccd8b0d615 Add tag exclusion support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 0
diff changeset
149 exclusion_tags = set(tag[1:] for tag in tags if tag[0] == '-')
63ccd8b0d615 Add tag exclusion support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 0
diff changeset
150
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
151 # Get the list of the files corresponding to those tags.
2
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
152 files = reduce((lambda s, t: s.intersection(self.tags[t])),
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
153 inclusion_tags, set(self.files))
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
154 files -= set([f for f in files
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
155 if exclusion_tags.intersection(self.files[f])])
1
63ccd8b0d615 Add tag exclusion support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 0
diff changeset
156
63ccd8b0d615 Add tag exclusion support.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 0
diff changeset
157 # Those next two steps are for useless tags removal.
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
158
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
159 # Get the tags of those files.
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
160 taglist = reduce((lambda s, f: s.union(self.files[f])), files, set())
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
161 taglist -= tags
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
162
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
163 # Remove the tags that can’t precise the file list anymore.
2
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
164 remove = reduce((lambda s, f: s.intersection(self.files[f])), files,
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
165 taglist)
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
166 taglist -= remove
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
167
2
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
168 self.cache[key] = list(taglist) + list(files)
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
169 return ['.', '..'] + self.cache[key]
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
170
4
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
171 def readlink(self, path):
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
172 _, filename = self._split_path(path)
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
173 return filename
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
174
2
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
175 def release(self, path, file_handle):
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
176 return os.close(file_handle)
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
177
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
178 def statfs(self, path):
2
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
179 _, filename = self._split_path(path)
85cbd44f98b1 Add license, (try to) respect the PEP 8, and don’t override the file builtin.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 1
diff changeset
180 path = filename if filename else self.root
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
181 stv = os.statvfs(path)
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
182 return dict((key, getattr(stv, key)) for key in ('f_bavail', 'f_bfree',
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
183 'f_blocks', 'f_bsize', 'f_favail', 'f_ffree', 'f_files', 'f_flag',
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
184 'f_frsize', 'f_namemax'))
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
185
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
186 utimens = os.utime
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
187
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
188
4
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
189 def main(args):
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
190 mountpoint = args.pop()
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
191
5
a422e75bf464 Default to symlinks mode, since it’s better.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 4
diff changeset
192 if args[1] == '-n' or args[1] == '--no-symlinks':
a422e75bf464 Default to symlinks mode, since it’s better.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 4
diff changeset
193 use_symlinks = False
6
2c81cc41de2d Allow only a base directory for the tags, which will be recursively walked to find the actual tags files.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 5
diff changeset
194 directory = args[2]
5
a422e75bf464 Default to symlinks mode, since it’s better.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 4
diff changeset
195 else:
4
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
196 use_symlinks = True
6
2c81cc41de2d Allow only a base directory for the tags, which will be recursively walked to find the actual tags files.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 5
diff changeset
197 directory = args[1]
2c81cc41de2d Allow only a base directory for the tags, which will be recursively walked to find the actual tags files.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 5
diff changeset
198
2c81cc41de2d Allow only a base directory for the tags, which will be recursively walked to find the actual tags files.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 5
diff changeset
199 filelist = []
2c81cc41de2d Allow only a base directory for the tags, which will be recursively walked to find the actual tags files.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 5
diff changeset
200 start = time()
2c81cc41de2d Allow only a base directory for the tags, which will be recursively walked to find the actual tags files.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 5
diff changeset
201 for (path, _, files) in os.walk(directory):
2c81cc41de2d Allow only a base directory for the tags, which will be recursively walked to find the actual tags files.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 5
diff changeset
202 filelist.extend(os.path.join(path, filename) for filename in files
2c81cc41de2d Allow only a base directory for the tags, which will be recursively walked to find the actual tags files.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 5
diff changeset
203 if filename.endswith('.tags'))
2c81cc41de2d Allow only a base directory for the tags, which will be recursively walked to find the actual tags files.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 5
diff changeset
204 print('[%d] Walk done.' % (time() - start))
4
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
205
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
206 FUSE(Danbooru(filelist, os.path.dirname(mountpoint), use_symlinks),
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
207 mountpoint, foreground=True)
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
208
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
209
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
210 if __name__ == '__main__':
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
211 if len(argv) < 3:
5
a422e75bf464 Default to symlinks mode, since it’s better.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 4
diff changeset
212 print('USAGE: %s' % argv[0], '[-n|--no-symlinks]',
6
2c81cc41de2d Allow only a base directory for the tags, which will be recursively walked to find the actual tags files.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 5
diff changeset
213 '<tags directory>', '<mountpoint>')
0
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
214 exit(1)
215d51f2a82f Initial commit.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents:
diff changeset
215
4
c40f0eed70cd Add a symlink mode, for performances reason.
Emmanuel Gil Peyrot <linkmauve@linkmauve.fr>
parents: 3
diff changeset
216 main(argv)