Mercurial > feed-push
changeset 1:a68d7feeba88 draft
initial commit
Signed-off-by: Changaco <changaco ατ changaco δοτ net>
| author | Changaco <changaco ατ changaco δοτ net> |
|---|---|
| date | Sun, 15 Apr 2012 18:05:20 +0200 |
| parents | 31c0a15dd43d |
| children | 292896092ee6 |
| files | README examples/blog.atom examples/pandoc.conf examples/sendmail.conf examples/sendxmpp-py.conf feed-push rc.d/feed-push setup |
| diffstat | 8 files changed, 836 insertions(+), 0 deletions(-) [+] |
line wrap: on
line diff
new file mode 100644 --- /dev/null +++ b/README @@ -0,0 +1,14 @@ +feed-push is a daemon that watches local RSS/Atom files for changes and executes commands when new articles appear. + +It can replace scripts such as rss2email and rss2xmpp, using tools like sendmail and sendxmpp.py (http://changaco.net/code/sendxmpp-py). + +Dependencies: +- gamin and its python2 bindings +- python2-feedparser + +Compatibility: gamin only works on Linux and FreeBSD as of April 2012 + +Installation: ./setup install +Removal: ./setup uninstall + +Configuration: see examples/
new file mode 100644 --- /dev/null +++ b/examples/blog.atom @@ -0,0 +1,421 @@ +<?xml version="1.0" encoding="utf-8"?> + +<feed xmlns="http://www.w3.org/2005/Atom"> +<title>Changaco's blog</title> +<link href="http://changaco.net/blog/"/> +<link href="http://changaco.net/blog/feed.atom" rel="self" type="application/atom+xml"/> +<author> +<name>Changaco</name> + +</author> + + + +<id>http://changaco.net/blog/</id> + +<subtitle type="html">Changaco</subtitle> +<generator uri="http://ikiwiki.info/" version="3.20111107">ikiwiki</generator> +<updated>2011-12-10T13:52:20Z</updated> +<entry> + <title>DNS problems and alternatives</title> + + <id>http://changaco.net/blog/DNS_problems_and_alternatives/</id> + + <link href="http://changaco.net/blog/DNS_problems_and_alternatives/"/> + + + + + + <updated>2011-12-10T13:34:21Z</updated> + <published>2011-12-10T13:34:21Z</published> + + <content type="html" xml:lang="en"> + <p>Replacing the <abbr title="Domain Name System">DNS</abbr> is a recurrent topic. In this post I try to explain the problems and give a list of existing or proposed alternatives.</p> + +<h2 id="problemsofthedns">Problems of the DNS</h2> + +<p>A little terminology first : the DNS has two functions, registering and resolving names. Critics of the registration mechanisms are mostly political, resolution problems are mostly technical.</p> + +<h3 id="censorship">Censorship</h3> + +<p>The US government has <a href="http://torrentfreak.com/feds-seize-130-domain-names-in-mass-crackdown-111125/">seized many domain names in November 2011</a>, as it had done the year before. Contrary to what some people said, <a href="http://domainincite.com/icann-had-no-role-in-seizing-torrent-domains/">the ICANN was not involved in those operations</a>. It was <a href="http://en.wikipedia.org/wiki/Verisign">Verisign</a>, the operator of the .com, .net, and .name generic top-level domains, that was ordered to seize the domains. As a result, some sites have fled generic TLDs controlled by US companies.</p> + +<h3 id="economicvampirismanddomainparking">Economic vampirism and domain parking</h3> + +<p>The DNS is a big profitable business.</p> + +<p>The name renting (you can't buy a domain name) business works like this : client → registrar (domain manager) → registry (<abbr title="Top-Level Domain">TLD</abbr> manager) → ICANN (root manager). Some of these organizations are nonprofit (e.g. ICANN), but that doesn't mean people working for them don't profit (there are high salaries, expensive dinners, trips, etc). Others are corporations that make very good profits<a href="http://changaco.net/blog/#fn:1" id="fnref:1" class="footnote">1</a>.</p> + +<p>X.509 certificates are another business. They are delivered by Certificate Authorities and used in TLS. This security model has been widely criticized<a href="http://changaco.net/blog/#fn:2" id="fnref:2" class="footnote">2</a><a href="http://changaco.net/blog/#fn:3" id="fnref:3" class="footnote">3</a><a href="http://changaco.net/blog/#fn:4" id="fnref:4" class="footnote">4</a> and there are plans to put certificates directly in DNS records<a href="http://changaco.net/blog/#fn:5" id="fnref:5" class="footnote">5</a><a href="http://changaco.net/blog/#fn:6" id="fnref:6" class="footnote">6</a>, and others to replace X.509 by OpenPGP<a href="http://changaco.net/blog/#fn:7" id="fnref:7" class="footnote">7</a>.</p> + +<p>Finally, there is the very annoying <a href="http://en.wikipedia.org/wiki/domain%20parking">domain parking</a> business.</p> + +<h3 id="technicalproblems">Technical problems</h3> + +<p>Being very old, the DNS also has technical weaknesses.</p> + +<p>The first is slow propagation of records because the DNS uses time-based caches.</p> + +<p>The second is that records are not stored in a P2P network, but by authoritative servers, which can be taken down by <abbr title="Denial of Service">DoS</abbr> attacks if they aren't sufficiently protected. This is rarely a problem in practice though.</p> + +<h2 id="whyhaventtheproblemsbeensolvedyet">Why haven't the problems been solved yet ?</h2> + +<p>Well, because different people want things that are contradictory. The problem is often known as <a href="http://en.wikipedia.org/wiki/Zooko%27s%20triangle">Zooko's triangle</a>, but there are in fact more than three desirable properties for identifiers :<a href="http://changaco.net/blog/#fn:8" id="fnref:8" class="footnote">8</a></p> + +<ul> +<li>We want to <strong>choose</strong> a <strong>unique</strong> and <strong>memorable</strong> name so we can communicate it to somebody else even if we don't have our computer with us at the moment. Some people who always have their smartphone with them may argue that this property is not important anymore, but not everybody has a smartphone.</li> +<li>We want a <strong>censorship-free</strong> system.</li> +<li>We want our <strong>trademarks</strong> to be registered only by us.</li> +<li>We want links between documents that are <strong>stable in time</strong>, the Web doesn't like broken URLs.</li> +<li>We want the registration process to be <strong>easy, fast and free of charge</strong>.</li> +<li>We want a name to be <strong>resolvable</strong> to an address, otherwise it's of no use to us.</li> +<li>We want names that are <strong>recoverable</strong> in case of hijacking or loss of credentials.</li> +</ul> + +<h2 id="existingorproposedalternatives">Existing or proposed alternatives</h2> + +<p>I can't help but start by my own DNS replacement proposal. <img src="http://changaco.net/blog/../smileys/smile.png" alt=":)" /> The <a href="http://changaco.net/ins/">Internet Naming System</a> acknowledges that there is no perfect solution and chooses to keep a central authority for name allocation. It makes censorship automatically detectable but not impossible.</p> + +<p>Projects for P2P registration of names :</p> + +<ul> +<li><a href="http://dot-bit.org/">Dot-BIT</a> (<a href="irc://irc.freenode.net/namecoin">#namecoin on freenode</a>) uses Bitcoin-like proof-of-work (which assumes that honest nodes have the majority of computing power)</li> +<li><a href="http://www.p2pns.org/">P2PNS</a> assumes that a vast majority of peers is honest</li> +<li><a href="http://lauren.vortex.com/archive/000787.html">IDONS: Internet Distributed Open Name System</a> (<a href="http://forums.gctip.org/forum-34.html">forum</a>) seems dead</li> +<li><a href="irc://irc.efnet.org/dns-p2p">#dns-p2p</a>, which used to have a wiki on dot-p2p.org, never gave anything and is dead</li> +</ul> + +<p>Technical solutions for improving resolution :</p> + +<ul> +<li><a href="http://beehive.systems.cs.cornell.edu/codons.php">CoDoNS</a></li> +<li><a href="http://huitema.wordpress.com/2011/01/03/a-simple-p2p-dns-proposal/">A simple P2P DNS proposal</a></li> +</ul> + +<p>Other projects :</p> + +<ul> +<li><a href="http://opennicproject.org/">OpenNIC</a> (<a href="irc://irc.freenode.net/opennic">#opennic on freenode</a>, <a href="http://lists.darkdna.net/mailman/listinfo">OpenNIC lists</a>) is an alternative root</li> +<li><a href="http://dns.telecomix.org/">Telecomix Censorship-proof DNS</a> (<a href="irc://irc.telecomix.org/dns">#dns on telecomix IRC</a>)</li> +</ul> + +<p>Other proposals :</p> + +<ul> +<li>on the <a href="http://lists.zooko.com/mailman/listinfo/p2p-hackers">p2p-hackers list</a> : +<ul> +<li><a href="http://lists.zooko.com/pipermail/p2p-hackers/2010-December/002598.html">Secure, decentralized DNS (a.k.a. solving Zooko's triangle)</a></li> +<li><a href="http://lists.zooko.com/pipermail/p2p-hackers/2010-December/002587.html">.p2p domain</a></li> +</ul></li> +<li><a href="http://roland.entierement.nu/blog/2010/10/02/for-a-truly-acentric-internet.html">For a truly acentric Internet</a>, proposes to abandon meaningful identifiers (an old proposition that comes back regularly)</li> +<li><a href="http://www.templetons.com/brad/dns/">Problems, Goals and a Fix for Domain Names</a>, proposed to only allow trademarks as TLDs</li> +</ul> + +<h2 id="referencesandcredits">References and credits</h2> + +<p>Thanks to Stéphane Bortzmeyer for helping with this post.</p> + +<div class="footnotes"> +<hr /> +<ol> + +<li id="fn:1"><p><a href="http://www.chemla.org/textes/voleur.html">Confessions d'un voleur</a> [fr]<a href="http://changaco.net/blog/#fnref:1" class="reversefootnote">&#160;&#8617;</a></p></li> + +<li id="fn:2"><p><a href="https://www.eff.org/deeplinks/2010/03/researchers-reveal-likelihood-governments-fake-ssl">New Research Suggests That Governments May Fake SSL Certificates</a><a href="http://changaco.net/blog/#fnref:2" class="reversefootnote">&#160;&#8617;</a></p></li> + +<li id="fn:3"><p><a href="https://docs.google.com/present/view?id=df9sn445_206ff3kn9gs">It's Time to Fix HTTPS</a><a href="http://changaco.net/blog/#fnref:3" class="reversefootnote">&#160;&#8617;</a></p></li> + +<li id="fn:4"><p><a href="http://lair.fifthhorseman.net/~dkg/tls-centralization/">Technical Architecture shapes Social Structure: an example from the real world</a><a href="http://changaco.net/blog/#fnref:4" class="reversefootnote">&#160;&#8617;</a></p></li> + +<li id="fn:5"><p><a href="http://tools.ietf.org/wg/dane/">DNS-based Authentication of Named Entities - IETF Working Group</a><a href="http://changaco.net/blog/#fnref:5" class="reversefootnote">&#160;&#8617;</a></p></li> + +<li id="fn:6"><p><a href="http://www.bortzmeyer.org/jres-dane-2011.html">Exposé sur les clés dans le DNS à JRES</a> [fr]<a href="http://changaco.net/blog/#fnref:6" class="reversefootnote">&#160;&#8617;</a></p></li> + +<li id="fn:7"><p><a href="http://web.monkeysphere.info/">The Monkeysphere Project</a><a href="http://changaco.net/blog/#fnref:7" class="reversefootnote">&#160;&#8617;</a></p></li> + +<li id="fn:8"><p><a href="http://www.bortzmeyer.org/no-free-lunch.html">Inventer un meilleur système de nommage: pas si facile</a> [fr]<a href="http://changaco.net/blog/#fnref:8" class="reversefootnote">&#160;&#8617;</a></p></li> + +</ol> +</div> + + </content> + + + <link rel="comments" href="/blog/DNS_problems_and_alternatives/#comments" type="text/html" /> + + + <link rel="comments" href="/blog/DNS_problems_and_alternatives/comments.atom" type="application/atom+xml" /> + +</entry> +<entry> + <title>Privacy and distant storage</title> + + <id>http://changaco.net/blog/Privacy_and_distant_storage/</id> + + <link href="http://changaco.net/blog/Privacy_and_distant_storage/"/> + + + + + + + <category term="tags/idea" /> + + <category term="tags/internet" /> + + + <updated>2010-09-02T11:12:33Z</updated> + <published>2010-08-30T22:45:55Z</published> + + <content type="html" xml:lang="en"> + <p>Some people seem to think that their data is only safe in their own homes. I agree that not keeping a local copy or storing unprotected personal documents on a machine you don't control are bad things. But I was reminded today (while trolling on <a href="http://www.numerama.com/">Numerama</a>, a French tech-related news site) that having them home doesn't make them safe from:</p> + +<ul> +<li>hardware failures such as hard drive breakdowns (although <a href="http://smartmontools.sourceforge.net/">smartmontools</a> may be able to alert you before it is too late)</li> +<li>disasters such as fire</li> +</ul> + +<p>Of course, if your home burns, loosing your files will be the least of your concerns, but if you know they are safe it is one less thing to worry about.</p> + +<p>Then I realized that having backups in different geographic places does not necessarily endanger your privacy, it just depends on how you do it. What you need is to encrypt and/or cut the data so that the people who will store it for you will not be able to read or exploit it (just like <a href="http://en.wikipedia.org/wiki/Freenet">Freenet</a> does for different reasons).</p> + +<p>So the next question is where to store it ? I came to see three possibilities:</p> + +<ul> +<li>pay for some storage service, might be necessary if you have a lot of data</li> +<li>share storage space with peers, this was my original thought</li> +<li>share storage space with family and/or friends, this the safest of the three and credit goes to Kaliko for suggesting it on the <a href="xmpp:utopians@muc.changaco.net?join">utopians chat room</a></li> +</ul> + +<p>I believe, like many others, that a good place for such sharing software is in <a href="http://en.wikipedia.org/wiki/residential%20gateway">residential gateway</a>s, maybe we'll see it implemented someday in the <a href="http://wiki.debian.org/FreedomBox">Freedom Box</a> ?</p> + + </content> + + + <link rel="comments" href="/blog/Privacy_and_distant_storage/#comments" type="text/html" /> + + + <link rel="comments" href="/blog/Privacy_and_distant_storage/comments.atom" type="application/atom+xml" /> + +</entry> +<entry> + <title>tabs vs spaces</title> + + <id>http://changaco.net/blog/tabs_vs_spaces/</id> + + <link href="http://changaco.net/blog/tabs_vs_spaces/"/> + + + + + + <updated>2011-04-12T09:23:05Z</updated> + <published>2010-06-01T15:09:22Z</published> + + <content type="html" xml:lang="en"> + <p>In this post I try to summarize the different points of view on the tabs versus spaces war.</p> + +<h2 id="decompositionoftheproblem">Decomposition of the problem</h2> + +<p>Firstly, you need to understand the difference between the <strong>tab key</strong> and the <strong>tab character</strong>. What you text editor does when you press the tab key is a matter of configuration and has nothing to do with the problem discussed here.</p> + +<p>Secondly, we need to distinguish <strong>indentation</strong> and <strong>alignment</strong>, this is explained in <a href="http://www.iovene.com/61">TABs vs Spaces. The end of the debate.</a> and shows why the historical rendering of tabs is not fit for alignment.</p> + +<h2 id="thesolutions">The solutions</h2> + +<h3 id="useonlyspaces">Use only spaces</h3> + +<p>This is the solution proposed by many and is notably exposed in <a href="http://www.jwz.org/doc/tabs-vs-spaces.html">Tabs versus Spaces: An Eternal Holy War.</a></p> + +<p>The obvious solution when dynamic doesn't work is to fall back to static. Using only spaces does indeed work for both indentation and aligning and you can configure most text editors to make it as easy as using tabs. So, what's wrong with it ? Here's a list :</p> + +<ul> +<li>you can't use proportional fonts</li> +<li>you can't easily change the indentation width</li> +<li>your files are larger</li> +</ul> + +<p>The two first points are all about freedom, maybe you don't like proportional fonts to code, but some people do.</p> + +<p>As to the third point, people usually reject it by saying that it doesn't matter nowadays because of disks capacity, network speed and compression. Still, I wanted to make a <em>quick and dirty</em> measure of the impact of the 4 spaces policy on python 2.6 on my system as of June 2010 ( done in zsh ) :</p> + +<pre><code># cd /usr/lib/python2.6 +# for f in **/*(/); do mkdir -p "../python2.6.spaces/&#036;f" "../python2.6.tabs/&#036;f"; done; +# for f in **/*.py; do cp "&#036;f" "../python2.6.spaces/&#036;f"; cp "&#036;f" "../python2.6.tabs/&#036;f"; done; +# du -h --max-depth=0 python2.6.* +43M python2.6.spaces +43M python2.6.tabs + +# cd ../python2.6.tabs +# sed 's/^\(\t*\) /\1\t/' -i **/*.py +# du -h --max-depth=0 ../python2.6.tabs +41M ../python2.6.tabs +# sed 's/^\(\t*\) /\1\t/' -i **/*.py +# du -h --max-depth=0 ../python2.6.tabs +39M ../python2.6.tabs + +... I did it two more times but the rounded number stayed 39M +</code></pre> + +<p>The result is that using 4 spaces instead of tabs makes files about 10% bigger. If you get a different result or tested something else than python 2.6 I invite you to post a comment.</p> + +<h3 id="usespacesforalignment">Use spaces for alignment</h3> + +<p>Since the problem with tabs is alignment, some people argue that you can use whatever you want for indentation as long as you use spaces for alignment. If you choose to use tabs, the indentation width is no longer an issue and most of the space waste goes away, but you still can't use proportional fonts.</p> + +<h3 id="elastictabstops">Elastic tabstops</h3> + +<p>This solution solves all the issues listed here and makes alignment easier. How ? By redefining the way the tab character is displayed. It's all explained in <a href="http://nickgravgaard.com/elastictabstops/">Elastic tabstops - a better way to indent and align code</a>. The downside is that text editors' code has to be modified.</p> + +<h2 id="myopinion">My opinion</h2> + +<p>I use tabs for indentation, spaces for alignment and I wish elastic tabstops were more widely known, implemented and used.</p> + + </content> + + + <link rel="comments" href="/blog/tabs_vs_spaces/#comments" type="text/html" /> + + + <link rel="comments" href="/blog/tabs_vs_spaces/comments.atom" type="application/atom+xml" /> + +</entry> +<entry> + <title>Réponse à Daniel Glazman</title> + + <id>http://changaco.net/blog/R%C3%A9ponse_%C3%A0_Daniel_Glazman/</id> + + <link href="http://changaco.net/blog/R%C3%A9ponse_%C3%A0_Daniel_Glazman/"/> + + + + + + <updated>2011-12-10T13:50:13Z</updated> + <published>2010-05-16T20:49:20Z</published> + + <content type="html" xml:lang="en"> + <p>Comme le dit le dicton populaire : « mieux vaut tard que jamais ». Je vais donc profiter de l'ouverture de mon blog pour répondre à un message de Daniel Glazman, mais ce billet concerne aussi Tristan Nitot.</p> + +<p>Commençons par poser le décor, l'histoire commence avec une <a href="http://standblog.org/blog/post/2010/04/02/Interview-Glazman-BlueGriffon">interview de Daniel Glazman par Tristan Nitot</a>. Ayant le Standblog dans mon lecteur de flux je lis l'article et trouvant étrange le modèle économique choisi par M. Glazman je décide de laisser un commentaire :</p> + +<blockquote> + <p>Encore et toujours en train d'essayer de vendre des copies, quand allez-vous comprendre que ce n'est pas un modèle viable ?<br /> + Voir entre autres : <a href="http://hcsoftware.sourceforge.net/jason-rohrer/freeDistribution.html">Free Distribution</a>.</p> +</blockquote> + +<p>J'avoue que c'était plutôt <em>trollesque</em>, j'aurais pu commencer par une question plus neutre sur le pourquoi du modèle économique choisi.</p> + +<p>C'est là que commencent les choses intéressantes. Tout d'abord, les commentaires étaient modérés à priori ce qui compliquait le débat en introduisant de longs délais, on a continué malgré tout, jusqu'à ce que M. Nitot ferme les commentaires, sans donner d'explication. <a href="http://changaco.net/blog/#note1" id="note1_c1" title="Aller &agrave; la note 1">1</a></p> + +<p>Puis quelqu'un qui est abonné au blog de M. Glazman m'envoie un message sur Jabber pour me signaler que celui-ci a publié un billet pour continuer le débat : <a href="http://www.glazman.org/weblog/dotclear/index.php?post/2010/04/07/Usual-suspects">Usual suspects - &lt;Glazblog/&gt;</a>.</p> + +<p>Je réponds donc là-bas, mais M. Glazman en a vite marre et ferme les commentaires à son tour, en laissant ce message :</p> + +<blockquote> + <p>A ce point, crevé par le jetlag, une crève monumentale et mon retour tardif de Paris hier soir, ce commentateur me les broute menu-menu. Ses arguments sont tellement nuls ("<cite>Si BlueGriffon était financé par Mozilla on ne serait pas en train de parler de tout ça</cite>" et si ma tante en avait...) que j'en ai franchement ras-le-bol d'offrir une tribune à cette personne ; je ferme donc les commentaires sur cet article mais je laisse sa prose, elle fera date...</p> +</blockquote> + +<p>Tout d'abord je tiens à faire remarquer que j'ai passé pas mal de temps à rédiger mes commentaires or la majorité de mes propos n'a pas eu de réponse, par exemple ma question dans mon dernier commentaire sur la fiabilité du modèle économique choisi :</p> + +<blockquote> + <p>je ne vois pas en quoi le modèle économique de la vente d'extensions propriétaires serait sûr. Par exemple si la communauté développe les mêmes en libre que ferez-vous ? La course aux fonctionnalités ?</p> +</blockquote> + +<p>Enfin, et c'est la motivation première de ce billet, je tiens à dénoncer cette pratique de la fermeture des commentaires totalement contraire à la liberté d'expression, il y a des façons plus civilisées de clore un débat, qu'il soit <em>trollesque</em> ou non.</p> + +<p><a id="note1" href="http://changaco.net/blog/#note1_c1">↑</a> <strong>Édit :</strong> on me rétorque que les commentaires sont fermés automatiquement aux bouts de 3 jours sur le Standblog. À cela je réponds deux choses :</p> + +<ul> +<li>je ne vois aucune mention de ça sur ledit blog, il me semble que la moindre des choses serait de le signaler</li> +<li>je suis autant en désaccord avec cette pratique qu'avec la fermeture manuelle</li> +</ul> + + </content> + + + <link rel="comments" href="/blog/Réponse_à_Daniel_Glazman/#comments" type="text/html" /> + + + <link rel="comments" href="/blog/Réponse_à_Daniel_Glazman/comments.atom" type="application/atom+xml" /> + +</entry> +<entry> + <title>About this blog</title> + + <id>http://changaco.net/blog/About_this_blog/</id> + + <link href="http://changaco.net/blog/About_this_blog/"/> + + + + + + + <category term="tags/clevercss" /> + + <category term="tags/css3" /> + + <category term="tags/html5" /> + + <category term="tags/ikiwiki" /> + + + <updated>2011-12-10T13:52:20Z</updated> + <published>2010-05-16T20:49:20Z</published> + + <content type="html" xml:lang="en"> + <p>This site is powered by <a href="http://ikiwiki.info">Ikiwiki</a>. It took me a while to set it up and I have some things to share.</p> + +<p>Firstly :</p> + +<ul> +<li><a href="http://changaco.net/blog/../templates/">templates</a> that use HTML5</li> +<li><a href="http://changaco.net/blog/../code/s2p2/">s2p2</a>, a bash script that pre-processes JavaScript and ( Clever ) CSS</li> +</ul> + +<p>About <a href="http://changaco.net/blog/../style/">style sheets</a> :</p> + +<ul> +<li>there is quite some of CSS3, like selectors and flexible box model</li> +<li>the biggest part is written in <a href="http://sandbox.pocoo.org/clevercss/">Clever CSS</a></li> +</ul> + +<p>For Clever CSS, a <a href="http://changaco.net/blog/../code/ccss.xml">ccss syntax file</a> for <a href="http://kate-editor.org/">kate</a>.</p> + +<p>Concerning <a href="http://changaco.net/blog/../script/">JavaScript</a> :</p> + +<ul> +<li>I got rid of the default ikiwiki.js as I don't support old browsers</li> +<li><a href="http://changaco.net/blog/../code/chtml/">chtml</a> creates real elements from HTML code contained in comments of the form <code>&lt;!--(CHTML) ... --&gt;</code></li> +</ul> + +<p>Everything has been written with technologies supported by the current versions of Firefox and Chromium as of April 2010. A conditional comment hides scripts and styles from all versions of IE. If you are using an up-to-date free browser and you encounter a problem send me a mail or even better a patch, I'll see what I can do.</p> + +<h2 id="aboutcomments">About comments</h2> + +<p>After a lot of thinking and some debate I decided to enable pseudo-anonymous comments only ( you can put a nickname and a website ). This means no OpenID and no user registration.</p> + +<p>I'll try to talk about that in a future post on Internet Identity.</p> + +<h2 id="criticsofikiwiki">Critics of ikiwiki</h2> + +<ul> +<li>Perl sux</li> +<li>I couldn't find a good way to build a site from multiple repositories</li> +<li>no threaded comments</li> +<li>discussion pages as crappy as traditional MediaWiki ones ( maybe even more )</li> +</ul> + + </content> + + + <link rel="comments" href="/blog/About_this_blog/#comments" type="text/html" /> + + + <link rel="comments" href="/blog/About_this_blog/comments.atom" type="application/atom+xml" /> + +</entry> + +</feed>
new file mode 100644 --- /dev/null +++ b/examples/pandoc.conf @@ -0,0 +1,5 @@ +# use --log-level DEBUG --flood to test this + +% pandoc -f html -t markdown --no-wrap + +*.atom
new file mode 100644 --- /dev/null +++ b/examples/sendmail.conf @@ -0,0 +1,7 @@ +% cat >in +% echo 'Subject: [{feed.title}] {entry.title}' >h; echo >>h +% cat h in | sendmail postmaster@localhost; r=$? +% rm h in +% exit $r + +*.atom
new file mode 100644 --- /dev/null +++ b/examples/sendxmpp-py.conf @@ -0,0 +1,3 @@ +% sendxmpp.py -s '[{feed.title}] {entry.title}' foo@example.net + +*.atom
new file mode 100755 --- /dev/null +++ b/feed-push @@ -0,0 +1,290 @@ +#!/usr/bin/env python2 +# -*- coding: utf-8 -*- + +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. + +import argparse +import calendar +from functools import partial, reduce +from glob import glob +import json +import os +from os.path import abspath +import shlex +from subprocess import Popen, PIPE, STDOUT +import sys +from syslog import * +import time + +import feedparser +import gamin + + +# Constants + +log_levels = ['DEBUG', 'INFO', 'NOTICE', 'WARNING', 'ERR', 'CRIT', 'ALERT', 'EMERG'] + +gamin_events = { + 1:'GAMChanged', 2:'GAMDeleted', 3:'GAMStartExecuting', 4:'GAMStopExecuting', + 5:'GAMCreated', 6:'GAMMoved', 7:'GAMAcknowledge', 8:'GAMExists', 9:'GAMEndExist' +} + + +# Generic utils + +concat = lambda l: reduce(list.__add__, l, []) + +def dict_append(d, k, v): + d.setdefault(k, []) + d[k].append(v) + + +# Logging + +def log(*args): + if len(args) == 1: + priority, msg = LOG_INFO, args[0] + elif len(args) == 2: + priority, msg = args + else: + return + priority = 7 - priority + if priority < global_args.log_level: + return + if global_args.syslog: + syslog(*args) + else: + sys.stderr.write(log_levels[priority]+': '+msg+'\n') + +def ignore_event(path, event): + log(LOG_DEBUG, 'ignoring event '+gamin_events.get(event, str(event))+' on '+path) + + +# Config parsing + +def parse_config_file(config_fd): + feeds_paths = config_to_feed_paths_to_commands[config_fd.name] = {} + cmd = None + next_cmd = [] + config_fd.seek(0) + for i, line in enumerate(config_fd): + line = line.strip() + if len(line) == 0 or line[0] == '#': + continue + if line[0] == '%': + next_cmd.append(line[1:].rstrip(';')) + elif not next_cmd: + log(LOG_ERR, 'missing command in file '+config_fd.name+' before line '+str(i)) + return + else: + cmd = '; '.join(next_cmd) + next_cmd = [] + for feed_path in glob(line): + feed_path = abspath(feed_path) + dict_append(feeds_paths, feed_path, cmd) + if not feed_path in path_to_feed_fd: + monitor.watch_file(feed_path, handle_feed_change) + log('successfully parsed config file '+config_fd.name) + + +# Gamin callbacks + +def handle_config_change(path, event): + path = abspath(path) + if os.path.isdir(path): + ignore_event(path, event) + elif not path in path_to_config_fd: + open_config(path, event) + elif event in [gamin.GAMChanged, gamin.GAMDeleted]: + update_config(path, event) + else: + ignore_event(path, event) + +def open_config(path, event): + if event in [gamin.GAMCreated, gamin.GAMExists]: + if (not path.endswith('.conf') or path[0] == '.') and not hasattr(global_args.config, 'read'): + return log('ignoring '+path+' (not a valid config file name)') + try: + config_fd = open(path) + except IOError as e: + return log('failed to open "'+line+'" '+str(e)) + path_to_config_fd[path] = config_fd + parse_config_file(config_fd) + else: + ignore_event(path, event) + +def update_config(path, event): + feeds_paths = set(concat(config_to_feed_paths_to_commands.values())) + if event == gamin.GAMChanged: + log('updating actions from modified config file '+config_fd.name) + parse_config_file(path_to_config_fd[path]) + elif event == gamin.GAMDeleted: + log('removing actions from deleted config file '+config_fd.name) + config_to_feed_paths_to_commands.pop(path) + path_to_config_fd.pop(path).close() + new_feeds_paths = set(concat(config_to_feed_paths_to_commands.values())) + for feed_path in feeds_paths.difference(new_feeds_paths): + monitor.stop_watch(feed_path) + if feed_path in path_to_feed_fd: + path_to_feed_fd.pop(feed_path).close() + +def handle_feed_change(path, event): + if path not in path_to_feed_fd: + if event in [gamin.GAMCreated, gamin.GAMExists, gamin.GAMChanged]: + try: + feed_fd = path_to_feed_fd[path] = open(path) + except IOError as e: + return log('failed to open "'+path+'": '+str(e)) + feed = feedparser.parse(feed_fd.read()) + handle_feed_change(path, gamin.GAMChanged) + else: + ignore_event(path, event) + elif event == gamin.GAMChanged: + feed_fd = path_to_feed_fd[path] + feed_fd.seek(0) + feed = feedparser.parse(feed_fd.read()) + for entry in reversed(feed.entries): + if entry.id in state['id_cache'].get(feed_fd.name, []) or \ + not global_args.flood and calendar.timegm(entry.published_parsed) < time.time() - 86400: + continue + for feed_path_to_commands in config_to_feed_paths_to_commands.values(): + for cmd in feed_path_to_commands.get(path, []): + run_command(format_cmd(cmd, feed=feed.feed, entry=entry), entry.content[0].value) + state['id_cache'][feed_fd.name] = [entry.id for entry in feed.entries] + save_state() + elif event == gamin.GAMDeleted: + path_to_feed_fd.pop(path).close() + else: + ignore_event(path, event) + +def save_state(): + global_args.state_file.truncate(0) + json.dump(state, global_args.state_file) + global_args.state_file.flush() + + +# Commands utils + +def format_cmd(cmd, **kwargs): + """The safe equivalent of str.format() for shell commands, meaning interpolated variables can't do shell injections (I hope).""" + r = u'' + for arg in shlex.split(cmd.encode('utf8')): + a = arg.decode('utf8') + b = a.format(**kwargs) + if a != b: + r += u" '" + b.replace(u"'", u'\'"\'"\'') + u"'" + else: + r += u' ' + arg + return r.lstrip() + +def run_command(cmd, input): + p = Popen(cmd, stdin=PIPE, stdout=PIPE, stderr=STDOUT, shell=True) + output = p.communicate(input.encode('utf8'))[0].decode('utf8') + if p.returncode != 0: + log(LOG_ERR, 'command failed: '+cmd+'\n'+output) + else: + log(LOG_INFO, 'successfully executed '+cmd) + log(LOG_DEBUG, '===== output:\n'+output) + + +# Argparse utils + +def AbsPath(next_type=None): + def f(s): + p = abspath(s) + if next_type is not None: + return next_type(p) + else: + return p + return f + +class Apply(argparse.Action): + def __init__(self, f, *args, **kwargs): + super(self.__class__, self).__init__(**kwargs) + self.f = f + def __call__(self, parser, namespace, values, option_string=None): + setattr(namespace, self.dest, self.f(values[0])) + +def Directory(s): + try: + os.listdir(s) + return s + except OSError as e: + raise argparse.ArgumentTypeError(str(e)) + +def File(flags): + def f(s): + try: + return os.fdopen(os.open(s, flags), 'w') + except OSError as e: + raise argparse.ArgumentTypeError(str(e)) + return f + +class First(argparse.Action): + def __call__(self, parser, namespace, values, option_string=None): + setattr(namespace, self.dest, values[0]) + +def FirstOf(*types, **kwargs): + kwargs.setdefault('error', 'argument "{}" is not valid') + def f(s): + for t in types: + try: + return t(s) + except: + pass + raise argparse.ArgumentTypeError(error.format(s)) + return f + + +# Main + +if __name__ == '__main__': + + p = argparse.ArgumentParser() + p.add_argument('config', type=FirstOf(AbsPath(argparse.FileType('r')), AbsPath(Directory), error='"{}" is neither a file nor a directory'), help='either a file or a directory') + p.add_argument('state_file', type=argparse.FileType('a+'), help='e.g. /var/lib/feed-push/state') + p.add_argument('--flood', default=False, action='store_true', help='push all articles on startup instead of ignoring the ones older than 24h (useful for debugging)') + p.add_argument('--fork', metavar='pid-file', nargs=1, type=File(os.O_WRONLY|os.O_CREAT|os.O_EXCL), action=First, help='useful in init scripts') + p.add_argument('--log-level', nargs=1, default=1, choices=log_levels, action=partial(Apply, log_levels.index), help='default is INFO') + p.add_argument('--syslog', default=False, action='store_true', help='log to syslog instead of stderr') + global_args = p.parse_args() + + if global_args.fork: + pid = os.fork() + if pid != 0: + global_args.fork.write(str(pid)) + exit(0) + + if global_args.syslog: + openlog(facility=LOG_DAEMON) + + state = {'id_cache': {}} + saved_state = global_args.state_file.read().strip() + if len(saved_state) > 0: + state.update(json.loads(saved_state)) + del saved_state + + monitor = gamin.WatchMonitor() + path_to_feed_fd = {} + path_to_config_fd = {} + config_to_feed_paths_to_commands = {} + if hasattr(global_args.config, 'read'): + os.chdir(os.path.dirname(global_args.config.name)) + monitor.watch_file(global_args.config.name, handle_config_change) + else: + os.chdir(global_args.config) + monitor.watch_directory(global_args.config, handle_config_change) + + while True: + monitor.handle_one_event()
new file mode 100644 --- /dev/null +++ b/rc.d/feed-push @@ -0,0 +1,58 @@ +#!/bin/bash + +. /etc/rc.conf +. /etc/rc.d/functions + +get_pid() { + pgrep -f $daemon_name | grep -v $$ +} +PID=$(get_pid) + +daemon_bin="/usr/bin/feed-push" +daemon_name=$(basename $daemon_bin) +pid_file="/var/run/$daemon_name.pid" + +data_dir=/var/lib/$daemon_name +conf_dir=/etc/$daemon_name + +case "$1" in + start) + stat_busy "Starting $daemon_name daemon" + if [ -z "$PID" ]; then + [ -f $pid_file ] && rm -f $pid_file + $daemon_bin "$conf_dir" "$data_dir/state" --fork $pid_file --syslog + if [ $? -gt 0 ]; then + stat_fail + exit 1 + else + add_daemon $daemon_name + stat_done + fi + else + stat_fail + exit 1 + fi + ;; + + stop) + stat_busy "Stopping $daemon_name daemon" + [ ! -z "$PID" ] && kill $PID &> /dev/null + if [ $? -gt 0 ]; then + stat_fail + exit 1 + else + rm -f $pid_file &> /dev/null + rm_daemon $daemon_name + stat_done + fi + ;; + + restart) + $0 stop + sleep 1 + $0 start + ;; + + *) + echo "usage: $0 {start|stop|restart}" +esac
new file mode 100755 --- /dev/null +++ b/setup @@ -0,0 +1,38 @@ +#!/bin/sh + +usage () { + echo "usage: [DESTDIR=/] [PREFIX=/usr] $(basename "$0") install|uninstall" && exit 1 +} + +remove () { + [ -e "$1" ] && rm -vrf --preserve-root "$1" +} + +install_dir () { + mkdir -p "$2" + cp -rv "$1" "$2" +} + +[ $# -eq 1 ] && action=$1 || usage + +[ -z "$DESTDIR" ] && DESTDIR=/ +[ -z "$PREFIX" ] && PREFIX=/usr +_PREFIX=${DESTDIR%%/}/${PREFIX##/} + +[ ! -w "$DESTDIR" ] && echo "You don't have write access on $DESTDIR" && exit 1 + +progname=feed-push + +if [ "$action" == install ]; then + install -v -m 755 -D $progname "$_PREFIX/bin/$progname" + install_dir examples "$_PREFIX/share/$progname" + [ -d /etc/rc.d ] && install_dir rc.d "${DESTDIR}etc" +elif [ "$action" == uninstall ]; then + remove "$PREFIX/bin/$progname" + remove "$PREFIX/share/$progname/examples" + remove "/etc/rc.d/$progname" +else + usage +fi + +exit 0