Technically Personal - programming

Testing multiple versions of Python in parallel

2025-07-21T00:00:00+00:00

I was reading my RSS feed when I came across Daniel Roy Greenfeld’s article</a> about using uv</a> to run unit tests on multiple versions of Python. Daniel writes:</p>

But what if I want to test a particular version of Python? Then I simple specify the version of Python to run the test:</p>

uv</span> run --python=3.13 --with pytest --with httpx pytest</span></span></code></pre>
Here’s where it gets fun. I can use a Makefile</code> (or a justfile</a>) to test on multiple Python
versions.</p>
testall</span>:</span>  ## Run all the tests for all the supported Python versions</span></span>
    uv run --python=3.10 --with pytest --with httpx pytest</span></span>
    uv run --python=3.11 --with pytest --with httpx pytest</span></span>
    uv run --python=3.12 --with pytest --with httpx pytest</span></span>
    uv run --python=3.13 --with pytest --with httpx pytest</span></span></code></pre></blockquote>
This is fantastic! I’ve used uv</code> for only a couple of projects, and already I cannot imagine ever
giving it up again.</p>
However, there’s one disadvantage to the technique Daniel describes above: every time uv switches
between Python versions, it will have to remove and replace the whole .venv</code> directory and
reinstall all dependencies.</p>
For example, this is what happens when I consecutively run the tests on a project with two different
python versions:</p>
$</span> uv run --python</span> 3.12</span> --extra test pytest</span></span>
Using</span> CPython</span> 3.12.11</span> interpreter at: ...</span></span>
Creating</span> virtual environment at: .venv</span></span>
Installed</span> 5</span> packages in 10ms</span></span>
...</span></span>
</span>
$</span> uv run --python</span> 3.13</span> --extra test pytest</span></span>
Using</span> CPython</span> 3.13.5</span> interpreter at: ...</span></span>
Removed</span> virtual environment at: .venv</span></span>
Creating</span> virtual environment at: .venv</span></span>
Installed</span> 5</span> packages in 8ms</span></span></code></pre>
This most likely isn’t much of a problem, because of how fast uv</code> is, but it’s still not ideal. In
addition, using Makefiles to run multiple commands sequentially is sooOOoOo boring, especially now
that we have computers with insane processor counts. Let’s run these commands in parallel!</p>
Caveat emptor</strong></em>: This presumes that your tests don’t have any real side-effects, and that they
can be run fully simultaneously without clashing with one another (shared databases like sqlite, or
creating test files could wreak havoc here).</p>
We need to deploy a few tricks so that uv</code> won’t stumble over itself and try to re-use the .venv</code>
directory. In the end, this ends up being slightly less elegant than I wanted, but I guess that’s a
lesson for me in believing slightly too much in Makefiles.</p>
.PHONY</span>:</span> testall</span></span>
.PRECIOUS</span>:</span> .venv-%/pyvenv.cfg</span></span>
</span>
PYTHON_VERSIONS</span> :=</span> 3.10 3.11 3.12 3.13</span></span>
TEST_TARGETS</span> :=</span> $(</span>addprefix</span> .tests-</span>,</span>$(PYTHON_VERSIONS))</span></span>
</span>
testall</span>:</span> $(TEST_TARGETS)</span></span>
</span>
.venv-%/pyvenv.cfg</span>:</span></span>
    uv venv \</span></span>
      --python $(</span>patsubst</span> .venv-%</span>,</span>%</span>,</span>$(@D)) \</span></span>
      $(@D)</span></span>
</span>
.tests-%</span>:</span> .venv-%/pyvenv.cfg</span></span>
    venv</span>=</span>$(</span>patsubst</span> .tests-%</span>,</span>.venv-%</span>,</span>$@) && </span>\</span></span>
    source $$venv/bin/activate && </span>\</span></span>
    uv run </span>\</span></span>
      --active </span>\</span></span>
      --python $$venv </span>\</span></span>
      --extra test </span>\</span></span>
      pytest</span></span>
    touch $@</span></span></code></pre>
Okay, there’s a lot to unpack here. Let’s dive in.</p>
The .PHONY</code> and .PRECIOUS</code> lines are easily explained: there will never be a file called
testall</code>, so no need to check if it exists or not, and don’t delete the intermediate files created
when we initiate a venv.</p>
Similarly easy to understand, the PYTHON_VERSIONS</code> just contains all the versions of Python we want
to test against. And the TEST_TARGETS</code> represents the actual targets in the file we want to
execute for (tests-3.13</code>, tests-3.12</code>, etc).</p>
The pyvenv.cfg</code> target just says “if anyone needs .venv-whatever/pyvenv.cfg</code>, here’s how to create
it”. It very basically calls uv venv</code> and initialises a virtualenv with the correct Python version.
The pyvenv.cfg</code> file is created by uv</code> when it sets up the virtualenv, satisfying the Make rule.</p>
The final target, .tests-%</code>, we create manually (touch $@</code>) after running uv run</code> inside the
proper virtualenv. This is most likely the part I’m the least happy with; I wish uv</code> had a way of
either prefixing venv dirs, or the ability to configure it to say “Python version 3.12 uses
.venv-3.12 as the venv dir” or something.</p>
But what does it do? Well, for starters, if you use it exactly like Daniel’s version, it behaves
exactly the same way, but it doesn’t replace .venvs all the time:</p>
$</span> make testall</span></span>
venv</span>=</span>.venv-3.10</span> &&</span> source</span> $venv</span>/bin/activate</span> &&</span> \</span></span>
  uv</span> run --active --python</span> $venv</span> --extra test pytest</span></span>
=============</span> 1</span> passed in 1.01s =============</span></span>
touch</span> .tests-3.10</span></span>
</span>
venv</span>=</span>.venv-3.11</span> &&</span> source</span> $venv</span>/bin/activate</span> &&</span> \</span></span>
  uv</span> run --active --python</span> $venv</span> --extra test pytest</span></span>
=============</span> 1</span> passed in 1.01s =============</span></span>
touch</span> .tests-3.11</span></span>
</span>
venv</span>=</span>.venv-3.12</span> &&</span> source</span> $venv</span>/bin/activate</span> &&</span> \</span></span>
  uv</span> run --active --python</span> $venv</span> --extra test pytest</span></span>
=============</span> 1</span> passed in 1.01s =============</span></span>
touch</span> .tests-3.12</span></span>
</span>
venv</span>=</span>.venv-3.13</span> &&</span> \</span> source</span> $venv</span>/bin/activate</span> &&</span> \</span></span>
  uv</span> run --active --python</span> $venv</span> --extra test pytest</span></span>
=============</span> 1</span> passed in 1.01s =============</span></span>
touch</span> .tests-3.13</span></span></code></pre>
But more interestingly, we can now run run all tests in parallel across versions of Python!</p>
$</span> time make testall</span></span>
make</span> testall  0.55s user 0.13s system 14% cpu</span> 4.725</span> total</span></span>
</span>
$</span> time make -j testall</span></span>
make</span> -j testall  0.63s user 0.17s system 64% cpu</span> 1.234</span> total</span></span></code></pre>
For real-world use, I would most likely also add a clean</code> target, and I would make the test targets
depend on the actual pyproject.toml</code> and .py</code> files, but that’s an exercise left for the reader.</p>


New project: comparqter
2025-07-01T00:00:00+00:00
Today I released comparqter</code> publicly</a>. It’s a small CLI tool that I’ve been
working on for a few weeks in my spare time that enables the compaction (and potentially
re-compression) of files in a datalake, typically hosted on S3.</p>
One of my clients had this issue, where their data lakes were filled with hundreds or thousands of
tiny files that contained anywhere between 1 and 2 rows, and sometimes a few hundred. This meant
that when Athena or another tool needed to query data, thousands upon thousands of files needed to
be scanned and analysed.</p>
This creates a massive issue in terms of throughput, but more importantly for them, it generates
massive amounts of HEAD and GET requests against S3, which ends up being quite expensive. From what
I could see in their AWS billing, I would expect that re-compacting their lakes would reduce their
ongoing S3 costs by about 20-40%.</p>
And now they have a tool to do it for free :).</p>
Well, the basis for a tool. This is just a quick first iteration—it is version 0.1.0 after all,
and it still needs a massive amount of extra testing. I have a few ideas of what could be
implemented next from a logical project perspective:</p>

Support getting configuration from env vars instead of CLI args</li>
Support for local file systems</li>
Support for other clouds

The AWS stuff would have to be gated behind a feature</li>
</ul>
</li>
Generate a report with the source files that were compacted</li>
Generate a report of the newly created files</li>
JSON logging, probably feature-gated

Improved logging</li>
</ul>
</li>
</ul>
And many more things! I’m happy to review any and all contributions, provided they aren’t just AI
slop. If you need a particular feature added, feel free to get in touch—I’m happy to be sponsored.</p>


The Date that wasn't
2025-05-03T00:00:00+00:00
Recently, I was working on a data project for a client that required diving into their data lake.
The last time I worked for this client, there was no data lake, so I just took everyone at face
value when it was suggested we use the lake to get the data we need. I’ve had quite a significant
amount of experience working with data lakes in the past few years, and the client is a modern
company, so I didn’t expect any issue. This is the same client that made me dust off my JVM and
Scala setup</a>.</p>
The data itself is event based, and spans about a decade. For the sake of visualisation, let’s say
that the events tracked visitors entering and exiting an amusement park. Pure events can be tricky
to work with in the real world, so we wanted to give the business daily aggregations.</p>
Two tables that were of interest to us for this project were admissions</code> and exits</code>. Both tables
had exactly the same schema:</p>

creation_time</code>: an ISO 8601-style UTC timestamp of when the event was generated</li>
company_id</code>: UUID4, the company a visitor is affiliated with, if any</li>
visitor_id</code>: UUID4 uniquely identifying the visitor</li>
</ul>
The lake itself is partitioned by date, so there’s an extra virtual column that appears in the
tables as dt</code> which is a string (due to HDFS partitions being part of the file path) representing a
specific date (for example: 2021-02-03</code>).</p>
We got to work and started writing some ETLs to get the required data. As a first draft, I quickly
wrote a SQL query that could be run on Athena to give us the data we needed:</p>
SELECT</span></span>
    dt,</span></span>
    SUM</span>(delta) </span>AS</span> visitors</span></span>
FROM</span> (</span></span>
    SELECT</span></span>
        dt,</span></span>
        company_id,</span></span>
        1</span> AS</span> delta</span></span>
    FROM</span> admissions</span></span>
    WHERE</span> dt </span>BETWEEN</span> '2021-01-01'</span> AND</span> '2021-12-31'</span></span>
</span>
    UNION ALL</span></span>
</span>
    SELECT</span></span>
        dt,</span></span>
        company_id,</span></span>
        -</span>1</span> AS</span> delta</span></span>
    FROM</span> exits</span></span>
    WHERE</span> dt </span>BETWEEN</span> '2021-01-01'</span> AND</span> '2021-12-31'</span></span>
) </span>AS</span> combined</span></span>
GROUP BY</span> dt, company_id</span></span>
ORDER BY</span> dt;</span></span></code></pre>
Note</strong>: Neither the data nor the queries are directly lifted from my client—I’m just retyping
from memory and trying to make it easier to grok. Some of these queries or code might not be working
exactly as is, but I hope it helps explain what we were doing.</p>
After running the query, we knew something was quite wrong: on some days we had a negative number of
visitors. I double checked the query, talked to the business to understand the behaviour of their
visitors, and everything was supposed to check out. We also checked non-negative days to see what
the potentially normal days were like. They were all completely wrong.</p>
I started digging more into the data. Because the dataset was reasonably large, I started scoping
down—instead of working on a whole year, I started looking month by month, but I couldn’t find a
specific pattern that made sense.</p>
While on a call with the other two engineers working on this, we weren’t sure how long it took for
events to replicate to the lake, so I ran a query to get the most recent partition:</p>
></span> SELECT</span> MAX</span>(</span>DATE</span>(dt)) </span>FROM</span> admissions;</span></span>
'2025-05-27'</span></span></code></pre>
Blink.</p>
I wrote that query around the end of April 2025, so seeing a date over a month in the future was a
bit of a shock. Was this lake actually tracking time travellers? Obviously not</a>, so
something had to be going on. I started digging in, and specifically looked at the single event
registered to this date. Its creation_time</code> was 2025-04-11T13:08:51.000Z</code>, or about two weeks
prior to when I was investigating this, but was otherwise unremarkable. Unremarkable, save for the
fact that two columns of the same row completely contradicted one another.</p>
So we started looking into the system that was writing these items to the lake. The system itself
was a Kafka consumer written in Scala. All three of us started poring over the code, but we couldn’t
see anything wrong. I obviously can’t share the actual code here, but in essence, each Kafka message
was an Avro record. The service decoded each record, enriched it with a bit of metadata, and then
wrote the entire thing to the lake as a Parquet file. Fairly unsurprising stuff. In pseudocode:</p>
object</span> AvroRecordTransformer</span> {</span></span>
</span>
  private val</span> dateFormatter</span> =</span> {</span></span>
    val</span> formatter</span> =</span> new</span> SimpleDateFormat</span>(</span>"yyyy-MM-dd"</span>)</span></span>
    formatter.setTimeZone(</span>TimeZone</span>.getTimeZone(</span>"UTC"</span>))</span></span>
    formatter</span></span>
  }</span></span>
</span>
  def</span> creationDateFrom</span>(</span>record</span>:</span> GenericRecord</span>)</span>:</span> Either</span>[</span>Throwable</span>, </span>String</span>]</span> =</span> Try</span> {</span></span>
    val</span> epochMillis</span> =</span> record.get(</span>"creationTime"</span>).</span>asInstanceOf</span>[</span>Long</span>]</span></span>
    val</span> date</span> =</span> new</span> Date</span>(epochMillis)</span></span>
    dateFormatter.format(date)</span></span>
  }</span>.toEither</span></span>
</span>
  def</span> toPartitionedMap</span>(</span>record</span>:</span> GenericRecord</span>)</span>:</span> Map</span>[</span>String</span>, </span>Any</span>]</span> =</span> {</span></span>
    val</span> baseMap</span>:</span> Map</span>[</span>String</span>, </span>Any</span>]</span> =</span> record.getSchema.getFields.asScala.map </span>{</span> field </span>=></span></span>
      val</span> name</span> =</span> field.name()</span></span>
      val</span> value</span> =</span> record.get(name)</span> match</span> {</span></span>
        case</span> utf8</span>:</span> Utf8</span> =></span> utf8.toString</span></span>
        case</span> nested</span>:</span> GenericRecord</span> =></span> toPartitionedMap(nested)</span></span>
        case</span> coll</span>: java.util.</span>Collection</span>[_]</span> =></span></span>
          coll.asScala.map </span>{</span></span>
            case</span> nested</span>:</span> GenericRecord</span> =></span> toPartitionedMap(nested)</span></span>
            case</span> primitive </span>=></span> primitive</span></span>
          }</span>.toList</span></span>
        case</span> other </span>=></span> other</span></span>
      }</span></span>
      name </span>-></span> value</span></span>
    }</span>.toMap</span></span>
</span>
    val</span> dtField</span>:</span> Option</span>[(</span>String</span>, </span>String</span>)]</span> =</span></span>
      creationDateFrom(record).toOption.map(</span>"dt"</span> -></span> _)</span></span>
</span>
    baseMap </span>++</span> dtField</span></span>
  }</span></span>
}</span></span></code></pre>
Did you catch it? I didn’t. It didn’t even occur to me in the least. I even talked about it with a
friend, and we went over the documentation of SimpleDateFormat</code> and we didn’t notice it.</p>
To be fair, it’s really well hidden, unless you know what to look for. It’s a class of problems I
had frankly almost forgotten about in the last few years, simply because I’ve been using languages
that don’t really have this kind of problem anymore.</p>
I was so sure the problem came from the asInstanceOf</code> line I even wrote this in the group chat:</p>
</p>
I thought that somehow, we were getting a weird value from the Avro message producers that was
causing the formatter to go haywire. In hindsight, it pains me to write down that this was where my
mind went, as it is a painfully obvious mistake. My brain kept hitting the same wall: if dt</code> and
creationTime</code> are based on the same value, how can dt</code> end up being completely different?</p>
We started grepping through the codebase, trying to find any producer that was generating these Avro
messages to see how they were initialising creationTime</code>. Most were using Instant.now, some were
truncating the millis, some were even truncating at the second level. Could there be something going
where instead of zeroing the least significant digits they were instead shifting the int to the
right? But that would only cause dates in the past, nothing in the future. We found nothing truly
odd in the message producers.</p>
I downloaded all the mispartitioned events in the lake1</a></sup>, and started analysing the dt</code> column.
Here are a few screenshots from my analysis with xan</code></a>2</a></sup>.</p>
</p>
During this analysis, I discovered an even weirder thing: some of the dt</code> values were invalid
dates. The first one I spotted was 2024-09-31</code>. The next ones were more usual suspects:
2025-02-29</code>, 2025-02-30</code> and 2025-02-31</code>, same for 2024. How could a battle-tested Java library
be generating invalid dates? Another odd thing is that we had dates all over 2025 and before, but
there was a hard stop on 2025-12-31</code>. Nothing beyond it.</p>
The issue was discovered late on Friday. We were digging through all the potential problems and
sifting through the data for a couple days. All of a sudden, Joana—one of the two brilliant
engineers working with me on this—asked: “what if it’s a thread safety problem?”</p>
We all stared at one another in stunned silence for a few seconds. My mind started racing: are there
thread unsafe parts in the Java standard library? How parallel is this code that writes to the lake?
How many pods are running?</p>
This is not a critical service, so unless it’s under very high load, only one pod will run. Joana
showed us what she was looking at: the code that reads the messages off the wire and feeds them into
the record transformer. It was using a parallel iterator to handle a bunch of messages
simultaneously.</p>
A few seconds later, we saw the warning in the legacy</em> java.text.SimpleDateFormat</code> documentation
that it is not thread safe. The comparatively modern java.time.format.DateTimeFormatter</code> is.</p>
It explained everything. The system could handle messages that weren’t exclusively instant. There
was backfilling happening, so we could continuously receive messages from months or years in the
past. Whenever multiple messages were handled together, the dt</code> would get botched in a way that
only thread-unsafety does. One message in late 2024 and one in early 2025 might partition the events
in late 2025. A message from the 31 August and another from 2 September might end up partitioned
into 31 September. Same for February.</p>
But because no message ever wrote 2026 onto the year part, even the thread-unsafe formatter couldn’t
invent that.</p>
Within minutes a fix was deployed. We had a bunch of logging that showed the severity of the
problem. We still have a slightly polluted lake to clean up, but at least we know there’s only clean
streams flowing into it.</p>
Do yourself a favour: grep your codebase for java.text.SimpleDateFormat</code>.</p>



It turns out it’s exceedingly trivial to write an Athena query that will make S3 very, very
unhappy with you. ↩</a></p>
</li>

xan</code></a> is an incredible</em> tool that you should definitely have in your tool belt if
you work with vast CSV files. It is ridiculously fast and is able to perform all the operations
you typically need to execute when doing exploratory data analysis. ↩</a></p>
</li>
</ol>
</section>


Rust and WASM for form validation
2025-04-24T00:00:00+00:00
In recent years, Rust and WebAssembly have become much more usable for pure backend-style engineers.
When I say “pure backend-style”, I mean people who never wrapped their heads around React, SPAs, and
all that stuff. This, unsurprisingly, includes me.</p>
For a very long time, in order to use WASM you were strongly guided towards using Webpack and a
whole array of Node-related tools in order to just get started. These days, luckily, the story has
become much more streamlined. In today’s tutorial, we’re going to create a quick project which
includes both a web server in Rust which renders HTML templates, and a small WASM component (served
by the same web server) that does form validation.</p>
Why?</h2>
Well, I want to keep a reference handy about how this can be done. But I also want to hear if I’m
doing something wrong or suboptimally.</p>
As to why Rust/WASM instead of standard JS or TS? For one, I like to see how far we’ve come with
WASM, and there is something very appealing about having the same code shared across the frontend
and the backend, especially if it means potentially having the same data structures when
serialising/deserialising data from one end to the other.</p>
I’m using form validation as a placeholder. It shows all the crucial aspects to use WASM instead of
JS, like wiring up DOM events to Rust functions, and then reacting to those events.</p>
Stack</h2>
This post is being written in early-to-mid 2025. And the stack I currently jive with is as follows:</p>

wasm-bindgen 0.2</li>
wasm-pack 0.13</li>
web-sys 0.3</li>
rocket 0.5</li>
</ul>
I’m not going to do any database-related things or anything, which allows us to keep the
dependencies pretty short and sweet. Most of these are library dependencies, however wasm-pack</code> is
a binary that actually modifies files and prepares them for distribution. This needs to be installed
locally:</p>
# binstall can also be used here</span></span>
cargo</span> install wasm-pack</span></span></code></pre>Project structure</h2>
Let’s create a directory structure that allows this whole setup to work. One typical approach for
this is to have separate crates for the server part, and the actual wasm component. Something like
this:</p>
tutorial</span></span>
├── server</span></span>
└── wasm</span></span></code></pre>
If you want to create things quickly:</p>
mkdir</span> tutorial</span> &&</span> cd</span> tutorial</span></span>
</span>
cargo</span> new --lib wasm</span></span>
cargo</span> new server</span></span></code></pre>
If you prefer to see the entire repo instead of creating all the files yourself, there is full
repository available here</a>.</p>
WASM setup</h2>
Installing wasm as a target</h3>
One thing that you might have to do is to get the compiler support to actually produce wasm
binaries. Provided you use rustup</a> (and you really ought to be using it), this is as simple
as:</p>
rustup</span> target add wasm32-unknown-unknown</span></span></code></pre>
You can check which targets you currently have installed with rustup target list --installed</code>.</p>
Configuring the wasm crate</h3>
Edit Cargo.toml</code> and add the following section. This tells the compiler to produce a dynamic
library that has a relatively stable ABI. This is similar to building a library for C/C++ interop.</p>
[</span>lib</span>]</span></span>
crate-type</span> =</span> [</span>"cdylib"</span>]</span></span></code></pre>
Let’s also pull in some dependencies while we’re editing that file:</p>
[</span>dependencies</span>]</span></span>
wasm-bindgen</span> =</span> "0.2"</span></span>
</span>
[</span>dependencies.web-sys</span>]</span></span>
version</span> =</span> "0.3"</span></span>
features</span> =</span> [</span></span>
  "Document"</span>,</span></span>
  "HtmlFormElement"</span>,</span></span>
  "HtmlInputElement"</span>,</span></span>
  "Window"</span>,</span></span>
  "console"</span>,</span></span>
  "Event"</span>,</span></span>
]</span></span></code></pre>
At this point, you should be able to build the library in wasm form using the following command:</p>
wasm-pack</span> build --target web</span></span></code></pre>
This will produce a few files in pkg/</code>:</p>
$</span> ls pkg/</span></span>
package.json</span>      wasm_bg.wasm      wasm_bg.wasm.d.ts</span></span>
wasm.d.ts</span>         wasm.js</span></span></code></pre>
Great, now that we’re producing a WASM library that can be loaded from a browser, it’s time to build
the server.</p>
Server side implementation</h2>
Let’s add a few libraries to our server’s Cargo.toml</code>:</p>
[</span>dependencies</span>]</span></span>
rocket</span> =</span> "0.5"</span></span>
</span>
[</span>dependencies.rocket_dyn_templates</span>]</span></span>
version</span> =</span> "0.2"</span></span>
features</span> =</span> [</span>"tera"</span>]</span></span></code></pre>
Rocket is our main server framework. It provides both sync and async support, has a strong database
integration, but also is easy enough to use that it won’t get in the way. In our src/main.rs</code>,
let’s add a some code to handle our fictional login. Anyone with the password “hunter2” will be
allowed in.</p>
fn</span> verify_password</span>(</span>password</span>: &</span>str</span>)</span> -></span> bool</span> {</span></span>
    password</span> ==</span> "hunter2"</span></span>
}</span></span>
</span>
#</span>[</span>get</span>(</span>"/login"</span>)]</span></span>
fn</span> render_login</span>()</span> -></span> Template</span> {</span></span>
    Template</span>::</span>render</span>(</span>"login"</span>,</span> context!</span> {})</span></span>
}</span></span>
</span>
#</span>[</span>derive</span>(</span>Debug</span>,</span> FromForm</span>)]</span></span>
struct</span> LoginInformation</span><</span>'</span>a</span>></span> {</span></span>
    email</span>: &</span>'</span>a</span> str</span>,</span></span>
    password</span>: &</span>'</span>a</span> str</span>,</span></span>
}</span></span>
</span>
#</span>[</span>post</span>(</span>"/login"</span>,</span> data </span>=</span> "<login>"</span>)]</span></span>
fn</span> handle_login</span>(</span>login</span>:</span> Form</span><</span>LoginInformation</span><</span>'</span>_</span>>></span>)</span> -></span> Template</span> {</span></span>
    if</span> verify_password</span>(</span>login</span>.</span>password</span>) {</span></span>
        Template</span>::</span>render</span>(</span></span>
            "success"</span>,</span></span>
            context!</span> {</span></span>
                username</span>:</span> login</span>.</span>email</span>,</span></span>
            },</span></span>
        )</span></span>
    }</span> else</span> {</span></span>
        render_login</span>()</span></span>
    }</span></span>
}</span></span></code></pre>
If you’ve worked with things like FastAPI the above most likely doesn’t shock you too much. We
define a verify_password</code> function that validates anyone who knows the secret handshake, and we
render a form where users can submit their credentials when they access /login</code>.</p>
We then define the shape of the form data that we expect (&'a str</code> is effectively a pointer to a
string, if you come from Golang or the Cs) in our LoginInformation struct, and we define a handler
when people submit a POST request to the same path /login</code>. Rocket handles the form parsing for us,
and will balk at the client if they submit unexpected data. When our function is called, we know the
data is at least UTF-8 which should be reasonably safe to process.</p>
Note</strong>: handle_login</code> could be declared as an async function (async fn handle_login(...)</code>), and
absolutely nothing else in our code would change. Rocket automatically handles the future that an
async function produces.</p>
We need a tiny bit more glue to start the rocket server, so let’s replace our main function with
something that actually starts the server:</p>
#</span>[</span>launch</span>]</span></span>
fn</span> rocket</span>()</span> -></span> _</span> {</span></span>
    rocket</span>::</span>build</span>()</span></span>
        .</span>mount</span>(</span>"/"</span>,</span> routes!</span>[</span>render_login</span>,</span> handle_login</span>])</span></span>
        .</span>mount</span>(</span>"/wasm"</span>,</span> FileServer</span>::</span>from</span>(</span>relative!</span>(</span>"../wasm/pkg"</span>)))</span></span>
        .</span>attach</span>(</span>Template</span>::</span>fairing</span>())</span></span>
}</span></span></code></pre>
There’s some cool stuff happening here. We tell Rocket that we want the /login</code> handlers to be
mounted on /</code>, and we start a file server handler on /wasm</code>, and we tell it to serve the files
created by our wasm library. Finally, we give Rocket a way to actually render our templates.</p>
We do need to create a couple of templates for Rocket to render our pages (both the login form and
the success page). I’m keeping things very simple and therefore ugly. I would advise young children
to close their eyes. Here is our beautiful login.html.tera</code>:</p>
<!DOCTYPE</span> html</span>></span></span>
<</span>html</span>></span></span>
  <</span>head</span>></span></span>
    <</span>meta</span> content</span>=</span>"text/html;charset=utf-8"</span> http-equiv</span>=</span>"Content-Type"</span>/></span></span>
    <</span>meta</span> name</span>=</span>"viewport"</span> content</span>=</span>"width=device-width, initial-scale=1"</span>></span></span>
    <</span>title</span>></span>Login</span></</span>title</span>></span></span>
  </</span>head</span>></span></span>
  <</span>body</span>></span></span>
    <</span>form</span> id</span>=</span>"login"</span> action</span>=</span>"/login"</span> method</span>=</span>"POST"</span>></span></span>
      <</span>div</span>></span></span>
        <</span>label</span> for</span>=</span>"email-field"</span>></span>Email</span></</span>label</span>></span></span>
        <</span>input</span> id</span>=</span>"email-field"</span> name</span>=</span>"email"</span> type</span>=</span>"email"</span></span>
               autocapitalize</span>=</span>"false"</span> placeholder</span>=</span>"johndoe@example.com"</span></span>
               autocomplete</span>=</span>"email"</span> required</span>></span></span>
      </</span>div</span>></span></span>
</span>
      <</span>div</span>></span></span>
        <</span>label</span> for</span>=</span>"password-field"</span>></span>Password</span></</span>label</span>></span></span>
        <</span>input</span> id</span>=</span>"password-field"</span> name</span>=</span>"password"</span> type</span>=</span>"password"</span></span>
               placeholder</span>=</span>"*******"</span> autocomplete</span>=</span>"password"</span></span>
               required</span>></span></span>
      </</span>div</span>></span></span>
</span>
      <</span>input</span> type</span>=</span>"submit"</span> value</span>=</span>"Login"</span>></span></span>
    </</span>form</span>></span></span>
</span>
    <</span>script</span> type</span>=</span>"module"</span>></span></span>
      import</span> init</span> from</span> './wasm/wasm.js'</span>;</span></span>
      init</span>()</span>;</span></span>
    <</span>/</span>script</span>></span></span>
  </</span>body</span>></span></span>
</</span>html</span>></span></span></code></pre>
And now for our success.html.tera</code>:</p>
<!DOCTYPE</span> html</span>></span></span>
<</span>html</span>></span></span>
  <</span>head</span>></span></span>
    <</span>meta</span> content</span>=</span>"text/html;charset=utf-8"</span> http-equiv</span>=</span>"Content-Type"</span>/></span></span>
    <</span>meta</span> name</span>=</span>"viewport"</span> content</span>=</span>"width=device-width, initial-scale=1"</span>></span></span>
    <</span>title</span>></span>Login successful</span></</span>title</span>></span></span>
  </</span>head</span>></span></span>
  <</span>body</span>></span></span>
    <</span>h2</span>></span>Welcome {{ username }}</span></</span>h2</span>></span></span>
    <</span>p</span>></span>You've successfully logged in!</span></span>
  </</span>body</span>></span></span>
</</span>html</span>></span></span></code></pre>
Shove both of these templates in a directory called templates</code> in your server</code> project, and Rocket
should be able to start when you launch it:</p>
$</span> cargo run</span></span>
...</span></span>
📬</span> Routes:</span></span>
   >></span> (</span>render_login</span>)</span> GET</span> /login</span></span>
   >></span> (</span>handle_login</span>)</span> POST</span> /login</span></span>
   >></span> (</span>FileServer:</span> ../wasm/pkg</span>)</span> GET</span> /wasm/</span><</span>path.</span>.</span>></span> [10]</span></span>
...</span></span>
📐</span> Templating:</span></span>
   >></span> directory:</span> templates</span></span>
   >></span> engines:</span> [</span>"tera"</span>]</span></span>
🚀</span> Rocket has launched from http://127.0.0.1:8000</span></span></code></pre>
Note</strong>: I’ve skipped a few lines of output from Rocket for brevity.</p>
What does this tell us? For one, Rocket properly picked up our route handlers and allows us to load
the wasm, the template engine is registered. With the server running, you should be able to access
the login form</a>.</p>
And with that, our server is done!</p>
Adding more features to the WASM</h2>
Now, form validation is pretty boring in and of itself; I just decided on that concept as a way to
demonstrate the usefulness of Rust. The first thing I’d like to do is make the WASM print something
in the console to show it did indeed correctly load. I think it’s quite poor practice to just
randomly leave messages in the browser console.log for no reason, but for this tutorial I think it’s
fine.</p>
In our wasm/src/lib.rs</code>, let’s replace the existing code with the following snippet:</p>
#</span>[</span>wasm_bindgen</span>(</span>start</span>)]</span></span>
pub fn</span> run</span>() {</span></span>
    web_sys</span>::</span>console</span>::</span>log_1</span>(</span>&</span>"Rust-based form validation starting up"</span>.</span>into</span>());</span></span>
}</span></span></code></pre>
After rebuilding the wasm library and ensuring the server is started, we can see the nice greeting
in our browser’s console. Success!</p>
Now, our HTML already does a lot of heavy lifting when it comes to form validation. If you just try
to login with “foo” as the email, your browser should prevent you from submitting the form and ask
for a valid email address. In my browser, the requirement is barely more than having an ‘@’ in the
value (so for example my browser will happilly accept .@e</code> as a valid email address).</p>
So what can we do to improve it?</p>
Well, modern browsers allow you to piggyback on top of their internal form validation. You don’t
need to add a whole bunch of extra HTML to display error messages and stuff. You just need to tell
the browser what to do.</p>
The first thing in that process is that we need to start doing some good ol’ getElementById stuff,
but oxidised. We need a handle on the form to be able to cancel its default behaviour, and submit it
later on once validation is complete.</p>
let</span> doc</span> =</span> window</span>()</span>.</span>unwrap</span>()</span>.</span>document</span>()</span>.</span>unwrap</span>();</span></span>
let</span> form</span> =</span> doc</span></span>
    .</span>get_element_by_id</span>(</span>"login"</span>)</span></span>
    .</span>unwrap</span>()</span></span>
    .</span>dyn_into</span>::</span><</span>HtmlFormElement</span>></span>()</span></span>
    .</span>unwrap</span>();</span></span>
let</span> form_ref</span> =</span> form</span>.</span>clone</span>();</span></span></code></pre>
The form_ref</code> is something we need because HtmlFormElement</code> is a refcounted object that can’t be
copied willy-nilly. And in the next section, we’ll move form</code> into a different context, but we
still need to keep a handle on it later.</p>
Next, we need to define the function that will be called by the onsubmit</code> event.</p>
let</span> closure</span> =</span> Closure</span>::</span><</span>dyn</span> Fn</span>(</span>Event</span>)</span>></span>::</span>new</span>(</span>move</span> |</span>event</span>:</span> Event</span>|</span> {</span></span>
    event</span>.</span>prevent_default</span>();</span></span>
</span>
    let</span> input</span> =</span> doc</span></span>
        .</span>get_element_by_id</span>(</span>"email-field"</span>)</span></span>
        .</span>unwrap</span>()</span></span>
        .</span>dyn_into</span>::</span><</span>HtmlInputElement</span>></span>()</span></span>
        .</span>unwrap</span>();</span></span>
</span>
    if</span> !</span>input</span>.</span>check_validity</span>() {</span></span>
        input</span>.</span>report_validity</span>();</span></span>
        return</span>;</span></span>
    }</span></span>
</span>
    if</span> Some</span>(</span>"example.com"</span>)</span> !=</span> input</span>.</span>value</span>()</span>.</span>split</span>('</span>@</span>')</span>.</span>nth</span>(</span>1</span>) {</span></span>
        input</span>.</span>set_custom_validity</span>(</span>"Email domain must be @example.com"</span>);</span></span>
        input</span>.</span>report_validity</span>();</span></span>
        input</span>.</span>set_custom_validity</span>(</span>""</span>);</span></span>
        return</span>;</span></span>
    }</span></span>
</span>
    form_ref</span>.</span>submit</span>()</span>.</span>unwrap</span>();</span></span>
});</span></span></code></pre>
Okay, there’s quite a bit going on here. Let’s break it down. The first line creates a new Closure
(a function that can be called JS/DOM). This closure must be 'static</code>, which is why we have that
move</code> keyword in there. Any outside variable we reference will be moved into it.</p>
Next, we do another get_element_by_id</code> to find our email input field, nothing surprising there. We
then use the constraints validation API</a> to, again, do the heavy lifting for us.
We start by checking whether the native checks pass, and report the error/refuse to submit if it
didn’t.</p>
The next chunk is bit more interesting, but still very basic. We know that the email at least
contains a ‘@’ because the native checks have passed, so we can just split the string on that. We
can then check if the user comes from the correct domain. If there is no domain to speak of (the
None</code> case), or if the domain is not example.com</code>, then we bail and show an error message.</p>
The slightly awkward call to set_custom_validity</code> with an empty string is sadly how this browser
API is expected to work.</p>
As a last step in this closure, if everything validated fine, we can submit the form.</p>
The final piece of the puzzle is to attach the closure to the form’s event listener. This can be
achieved with:</p>
form</span>.</span>add_event_listener_with_callback</span>(</span>"submit"</span>,</span> closure</span>.</span>as_ref</span>()</span>.</span>unchecked_ref</span>())</span>?</span>;</span></span>
closure</span>.</span>forget</span>();</span></span></code></pre>
This is why we had to create that form_ref</code> value that was used in the closure. We need both the
closure and the form, but the closure uses the form to submit, and then we need the form to attach
the closure. It’s a chicken and egg problem, so we get around it by cloning the egg. Or maybe we
cloned the chicken?</p>
Closing thoughts</h2>
I think that WASM story in Rust is really strong now. With just a handful lines of code we can get
access to an incredibly powerful language embedded in our otherwise extremely simple web page. WASM
is still relatively heavy1</a></sup>, compared to the equivalent JS that would’ve accomplished the same
(20KB or so vs a few hundred bytes), but this is also an extremely contrived example case. We’re
paying a heavy penalty to get started with WASM, because we are using a few libraries that hide a
lot of the ugly details from us. If we were to add a few hundred more validation functions, the code
size would barely increase, whereas it would increase linearly with JS, so there are economies of
scale to be had.</p>
Obviously, the sample code above unwrap</code>s to high heaven, and that’s nothing something I would
condone in actual production code—please do use proper error handling.</p>
That about wraps up our exercise for today. We’ve built a simple web app that uses Rust for both
backend and frontend logic, all powered by Rocket and WebAssembly. As a reminder, a full example of
the above tutorial is available on Codeberg</a>. If you try something similar, I’d love
to hear about it—and if there’s something deeply flawed in my approach, I’d be happy to be
corrected. Thanks for reading!</p>



It is possible to reduce the size of WASM binaries by using LTO, specific opt-levels, or even
using post-processing tools such as wasm-opt. Another trick is to get rid of the default
allocator if possible, or swap it out for a more lightweight one. These techniques are very
nicely documented in the Rust WASM book</a>, but even more so
in the min-sized-rust</a> repo. Thanks to Mendy for letting me
know</a> about this more up-to-date source. ↩</a></p>
</li>
</ol>
</section>


Getting started with Scala on macOS in 2025
2025-03-12T00:00:00+00:00
Scala</a> is a functional programming language that I’ve come across in the past, but it
never stuck in my tool belt. One of my clients has asked me to help them with a billing project, and
as this is a Scala shop, I figured it would make sense to get a head start and setup my workstation
to be ready to work with Scala again.</p>
The best way to get started with Scala is by using Coursier</a>; an application and artifact
manager for Scala. The official docs for both Scala</a> and
Coursier</a> seem to provide incorrect</del>suboptimal information for installation.
Both sources recommend installing coursier/formulas/coursier</code> using brew</code>, however that formula seems
to install an Intel binary, which obviously doesn’t run on Apple silicon.
Instead, the following workaround</a> appears to work
quite well:</p>
brew</span> install coursier</span></span>
coursier</span> setup</span></span></code></pre>
Once you have coursier installed and setup, you should have a fully functional Scala build
environment. Here’s the usual sanity check:</p>
@</span>main </span>def</span> hello</span>()</span> =</span> println(</span>"Hello, World!"</span>)</span></span></code></pre>
Which you can run with scala run hello.scala</code>. The output should look something like:</p>
Compiling project (Scala 3.6.4, JVM (23))</span></span>
Compiled project (Scala 3.6.4, JVM (23))</span></span>
Hello, World!</span></span></code></pre>
Fantastic. Now what’s left to do is to enable code completion for Scala in nvim. :LazyExtras</code>
should offer scala as a suggested language if you’re editing the hello.scala</code> file above, after
which a :MetalsInstall</code> should install everything needed to get up and running.</p>
Once Metals is installed, you should be able to see proper code completion. Sadly, type hints in the
form of inlays doesn’t seem to be supported. This is an extremely useful feature that I’ve come to
absolute love when writing Rust, and I don’t understand that it’s not a priority for most LSPs. I
find it almost universally useful, regardless of the language.</p>


Freebox automatic ruTorrent download
2016-04-26T00:00:00+00:00
I love Free. I think it is one of the best ISPs in the world, and even more so in France. If I’m not
mistaken, they invented the ISP-provided set top box that seems so common place these days. Their
Wikipedia entry</a> explains some of the reasons I like them so much, but to list a few:</p>

unmetered bandwidth,</li>
fixed IPv4 address, /60 IPv6 block,</li>
they allowed and encouraged people to run their own servers (reverse-IP customisation, every port
available, etc),</li>
their router doesn’t suck,</li>
fixed price policy: ~30 euros, all inclusive,</li>
the IPTV streams are viewable on your PCs through VLC,</li>
the list could go on for a long time.</li>
</ul>
All this being said, one of the major reason I loved their product so much was because of their
modem, the Freebox</a>. I’ve been a long fan of all their hardware (since the V4), but
the Freebox Revolution takes the cake.</p>
Anyway, back on topic. The Freebox Revolution Server allows you to set an RSS feed, and all items in
this RSS feed will be automatically downloaded by the newsgroup/torrent/direct download client that
is built into the router (yeah, they have that built in).</p>
I used to use a seedbox, but because I had a really bad DSL connection (4 Mbits), I preferred for my
downloads to happen during the day—when nobody was home—rather than having to painstakingly wait
for the bits to transfer when I was home. The other reason was also that downloading anything would
immediately saturate the link, and ruin browsing/YouTubing for anyone home at that point. The other
main point that should be stressed is that the Freebox Player (the part connected to the TV) could
stream any file stored on the Server’s hard drive (it acted as a NAS, out of the box). This meant
that we were able to watch the new GoT episode faster than most of my colleagues who had FTTH
connections, simply because we just had to turn the telly on, and click “Play”.</p>
I came up with the following script to generate an RSS feed of downloaded</em> files that my Freebox
would then automatically pick up.</p>
#!/usr/bin/python</span></span>
</span>
import</span> cPickle</span> as</span> pickle</span></span>
import</span> xml</span>.</span>etree</span>.</span>cElementTree</span> as</span> et</span></span>
# You can find PyRSS2Gen at:</span></span>
# https://pypi.python.org/pypi/PyRSS2Gen</span></span>
import</span> PyRSS2Gen</span> as</span> RSS2</span></span>
import</span> urllib</span>,</span> mimetypes</span></span>
from</span> os</span> import</span> path</span>,</span> walk</span></span>
from</span> datetime</span> import</span> datetime</span></span>
</span>
DIRECTORY</span> =</span> '/home/rtorrent/downloaded'</span></span>
DATABASE</span> =</span> '/home/rtorrent/.rssdb'</span></span>
EXTENSIONS</span> =</span> [</span>'.avi'</span>,</span> '.mkv'</span>,</span> '.mp4'</span>,</span> '.mpeg'</span>,</span> '.mpg'</span>]</span></span>
HOSTNAME</span> =</span> 'myseedbox.example.com'</span></span>
PATH</span> =</span> 'downloads'</span></span>
HTTPS</span> =</span> True</span></span>
RSS_FILENAME</span> =</span> '/home/rtorrent/downloads.rss'</span></span>
</span>
class</span> FileList</span>:</span></span>
  def</span> __init__</span>(</span>self</span>,</span> database</span>):</span></span>
    self</span>.</span>database</span> =</span> database</span></span>
    self</span>.</span>load</span>()</span></span>
</span>
  def</span> append</span>(</span>self</span>,</span> file</span>,</span> size</span>,</span> type</span>):</span></span>
    if</span> file</span> not in</span> self</span>.</span>files</span>:</span></span>
      self</span>.</span>files</span>[</span>file</span>]</span> =</span> {</span></span>
        'date'</span>:</span> datetime</span>.</span>now</span>(),</span></span>
        'size'</span>:</span> size</span>,</span></span>
        'mimetype'</span>:</span> type</span></span>
      }</span></span>
</span>
  def</span> store</span>(</span>self</span>):</span></span>
    with</span> open</span>(</span>self</span>.</span>database</span>,</span> 'w'</span>)</span> as</span> fp</span>:</span></span>
      pickle</span>.</span>dump</span>(</span>self</span>.</span>files</span>,</span> fp</span>)</span></span>
</span>
  def</span> load</span>(</span>self</span>):</span></span>
    if</span> path</span>.</span>exists</span>(</span>self</span>.</span>database</span>):</span></span>
      with</span> open</span>(</span>self</span>.</span>database</span>,</span> 'r'</span>)</span> as</span> fp</span>:</span></span>
        self</span>.</span>files</span> =</span> pickle</span>.</span>load</span>(</span>fp</span>)</span></span>
    else</span>:</span></span>
      self</span>.</span>files</span> =</span> {}</span></span>
</span>
  def</span> get</span>(</span>self</span>):</span></span>
    """</span></span>
    Return the files in reverse order in which we stored them.</span></span>
    """</span></span>
    return</span> self</span>.</span>files</span></span>
</span>
class</span> DirectoryParser</span>:</span></span>
  def</span> __init__</span>(</span>self</span>,</span> directory</span>,</span> extensions</span> =</span> []):</span></span>
    self</span>.</span>directory</span> =</span> directory</span></span>
    self</span>.</span>extensions</span> =</span> extensions</span></span>
    mimetypes</span>.</span>init</span>()</span></span>
</span>
  def</span> parse</span>(</span>self</span>,</span> store</span>):</span></span>
    for</span> _path</span>,</span> directories</span>,</span> files</span> in</span> walk</span>(</span>self</span>.</span>directory</span>):</span></span>
      for</span> file</span> in</span> files</span>:</span></span>
        if</span> self</span>.</span>filter</span>(</span>file</span>):</span></span>
          type</span> =</span> self</span>.</span>get_filetype</span>(</span>file</span>)</span></span>
          full_path</span> =</span> path</span>.</span>join</span>(</span>_path</span>,</span> file</span>)</span></span>
          size</span> =</span> path</span>.</span>getsize</span>(</span>full_path</span>)</span></span>
          storeable_path</span> =</span> full_path</span>[</span>len</span>(</span>self</span>.</span>directory</span>)</span> +</span> 1</span>:]</span></span>
          store</span>.</span>append</span>(</span>storeable_path</span>,</span> size</span>,</span> type</span>)</span></span>
</span>
  def</span> filter</span>(</span>self</span>,</span> file</span>):</span></span>
    if</span> len</span>(</span>self</span>.</span>extensions</span>)</span> is</span> 0</span>:</span></span>
      return</span> True</span></span>
</span>
    elif</span> path</span>.</span>splitext</span>(</span>file</span>)[</span>1</span>]</span> in</span> self</span>.</span>extensions</span>:</span></span>
      return</span> True</span></span>
</span>
    else</span>:</span></span>
      return</span> False</span></span>
</span>
  def</span> get_filetype</span>(</span>self</span>,</span> file</span>):</span></span>
    return</span> mimetypes</span>.</span>types_map</span>[</span>path</span>.</span>splitext</span>(</span>file</span>)[</span>1</span>]]</span></span>
</span>
class</span> RSSGenerator</span>:</span></span>
  def</span> __init__</span>(</span>self</span>):</span></span>
</span>
    if</span> HTTPS</span>:</span></span>
      self</span>.</span>url</span> =</span> 'https://'</span></span>
    else</span>:</span></span>
      self</span>.</span>url</span> =</span> 'http://'</span></span>
    self</span>.</span>url</span> +=</span> '</span>%s</span>/</span>%s</span>/</span>%%</span>s'</span> %</span> (</span>HOSTNAME</span>,</span> PATH</span>)</span></span>
</span>
  def</span> generate</span>(</span>self</span>,</span> files</span>):</span></span>
    items</span> =</span> []</span></span>
</span>
    for</span> file</span> in</span> files</span>:</span></span>
      url</span> =</span> self</span>.</span>url</span> %</span> urllib</span>.</span>quote</span>(</span>file</span>)</span></span>
      item</span> =</span> RSS2</span>.</span>RSSItem</span>(</span></span>
        title</span> =</span> file</span>,</span></span>
        link</span> =</span> url</span>,</span></span>
        guid</span> =</span> RSS2</span>.</span>Guid</span>(</span>url</span>),</span></span>
        pubDate</span> =</span> files</span>[</span>file</span>][</span>'date'</span>],</span></span>
        enclosure</span> =</span> RSS2</span>.</span>Enclosure</span>(</span>url</span>,</span> files</span>[</span>file</span>][</span>'size'</span>],</span> files</span>[</span>file</span>][</span>'mimetype'</span>])</span></span>
      )</span></span>
      items</span>.</span>append</span>(</span>item</span>)</span></span>
</span>
    self</span>.</span>rss</span> =</span> RSS2</span>.</span>RSS2</span>(</span></span>
      title</span> =</span> '</span>%s</span> downloads RSS feed'</span> %</span> HOSTNAME</span>,</span></span>
      description</span> =</span> 'The latest finished downloads at </span>%s\'</span>s ruTorrent'</span> %</span> HOSTNAME</span>,</span></span>
      link</span> =</span> self</span>.</span>url</span> %</span> ''</span>,</span></span>
      items</span> =</span> items</span></span>
    )</span></span>
</span>
  def</span> store</span>(</span>self</span>,</span> filename</span>):</span></span>
    with</span> open</span>(</span>filename</span>,</span> 'w'</span>)</span> as</span> fp</span>:</span></span>
      self</span>.</span>rss</span>.</span>write_xml</span>(</span>fp</span>)</span></span>
</span>
if</span> __name__</span> ==</span> '__main__'</span>:</span></span>
  file_list</span> =</span> FileList</span>(</span>DATABASE</span>)</span></span>
  parser</span> =</span> DirectoryParser</span>(</span>DIRECTORY</span>,</span> extensions</span> =</span> EXTENSIONS</span>)</span></span>
  parser</span>.</span>parse</span>(</span>file_list</span>)</span></span>
</span>
  file_list</span>.</span>store</span>()</span></span>
</span>
  rss_generator</span> =</span> RSSGenerator</span>()</span></span>
  rss_generator</span>.</span>generate</span>(</span>file_list</span>.</span>get</span>())</span></span>
</span>
  rss_generator</span>.</span>store</span>(</span>RSS_FILENAME</span>)</span></span></code></pre>
This was called by a cron job every 5 minutes:</p>
# Generate the downloads.rss, every 5 minutes</span></span>
*</span>/5 </span>* * * *</span> /usr/local/bin/rssdirectory.py </span>&&</span> /bin/sed</span> -i 's!></enclosure!/!g' /home/rtorrent/downloads.rss</span></span></code></pre>
The reason that sed command is there is because the Freebox doesn’t understand the enclosure tag
that is generated by PyRss2Gen. Yeah, that one took a while to debug and figure out.</p>
While this setup worked very nicely, every couple of months, the Freebox would simply stop
downloading new items. In the router’s WebOS, I’d force updates to the RSS feed, but nothing would
show up. The feed would be empty. After some time, I figured out that it was due to the Freebox not
being able to cope with the size of the RSS file. So here’s the hack-ish remedy for that, you’ve
guessed it, another cron job:</p>
# Remove the .rssdb once a week (garbage collection)</span></span>
@weekly</span> /bin/rm /home/rtorrent/.rssdb</span></span></code></pre>
This setup worked very nicely for the better part of 5 years, while we lived in France.</p>
So thank you Free, and all the engineers behind the amazing ISP that you are, and all the engineers
who make the Freebox possible. It’s an amazing piece of kit, and the hacker spirit that you guys
show all over the place is awe-inspiring, to say the least.</p>


A reply for Tim's StackOverflow question
2015-08-25T00:00:00+00:00
An old Maemo friend reached out on Twitter</a> and asked for help regarding some issue he was
running into using mysqli and PHP. Below is my response that I didn’t have time to post before the
question</a> was locked.</p>
Hi Tim,</p>
Looks like you’ve had a few issues here :). As a general rule, I like to go back to pure-MySQL, and
remove as much application logic as possible. In this case, as you’ve already discovered, it’s
possible to do it all in MySQL.</p>
But first, let’s do part of what you forgot to do (and which is probably why you got downvoted a
bit): let’s create a test MySQL table so we can run some queries on it:</p>
mysql</span>></span> create table</span> uploads (</span></span>
    -></span>   af_uid </span>int</span> unsigned,</span></span>
    -></span>   af_fid </span>int</span> unsigned,</span></span>
    -></span>   af_dfilename </span>varchar</span>(</span>100</span>),</span></span>
    -></span>   af_upload_date </span>datetime</span></span>
    -></span> );</span></span>
</span>
mysql</span>></span> insert into</span> uploads </span>values</span></span>
    -></span> (</span>101</span>, </span>10</span>, </span>'cat.jpg'</span>, </span>'2015-08-16 14:42:46'</span>),</span></span>
    -></span> (</span>101</span>, </span>10</span>, </span>'dog.jpg'</span>, </span>'2015-08-16 14:43:01'</span>),</span></span>
    -></span> (</span>101</span>, </span>11</span>, </span>'doc.pdf'</span>, </span>'2015-08-16 14:44:23'</span>),</span></span>
    -></span> (</span>101</span>, </span>10</span>, </span>'foo.jpg'</span>, </span>'2015-08-16 14:45:00'</span>),</span></span>
    -></span> (</span>101</span>, </span>10</span>, </span>'bar.jpg'</span>, </span>'2015-08-16 14:45:36'</span>),</span></span>
    -></span> (</span>101</span>, </span>10</span>, </span>'php.jpg'</span>, </span>'2015-08-16 14:46:10'</span>),</span></span>
    -></span> (</span>101</span>, </span>10</span>, </span>'mysql.jpg'</span>, </span>'2015-08-16 14:46:52'</span>),</span></span>
    -></span> (</span>102</span>, </span>10</span>, </span>'fubar.jpg'</span>, </span>'2015-08-16 14:51:03'</span>);</span></span></code></pre>
From your question, there’s 4 parts to the query you’re trying to write:</p>

you only want uid = 101</code>;</li>
you only want fid = 10</code>;</li>
you want the 5 last items uploaded;</li>
you only want the 5 first results.</li>
</ul>
mysql</span>></span> select</span> *</span></span>
    -></span> from</span> uploads</span></span>
    -></span> where</span> af_uid </span>=</span> 101</span></span>
    -></span> and</span> af_fid </span>=</span> 10</span></span>
    -></span> order by</span> af_upload_date </span>desc</span></span>
    -></span> limit</span> 5</span>;</span></span>
+</span>--------+--------+--------------+---------------------+</span></span>
| af_uid | af_fid | af_dfilename | af_upload_date      |</span></span>
+</span>--------+--------+--------------+---------------------+</span></span>
|    </span>101</span> |     </span>10</span> | mysql.jpg    | </span>2015</span>-</span>08</span>-</span>16 14</span>:</span>46</span>:</span>52</span> |</span></span>
|    </span>101</span> |     </span>10</span> | php.jpg      | </span>2015</span>-</span>08</span>-</span>16 14</span>:</span>46</span>:</span>10</span> |</span></span>
|    </span>101</span> |     </span>10</span> | bar.jpg      | </span>2015</span>-</span>08</span>-</span>16 14</span>:</span>45</span>:</span>36</span> |</span></span>
|    </span>101</span> |     </span>10</span> | foo.jpg      | </span>2015</span>-</span>08</span>-</span>16 14</span>:</span>45</span>:</span>00</span> |</span></span>
|    </span>101</span> |     </span>10</span> | dog.jpg      | </span>2015</span>-</span>08</span>-</span>16 14</span>:</span>43</span>:</span>01</span> |</span></span>
+</span>--------+--------+--------------+---------------------+</span></span>
5</span> rows in set</span> (</span>0</span>.</span>01</span> sec)</span></span></code></pre>
I’m not seeing limit</code> misbehaving here, as long as the rest of the query does what it’s supposed to
do. A couple notes to keep in mind for the future, though:</p>
Please don’t put integer values in quotes. In SQL (and most other languages), there’s no reason to
do so. You’re just asking the system to do more casting and whatnot. Secondly, try to do most work
in MySQL, especially when it comes to filtering. Always try to reduce your dataset as much as
possible in the first step. There are certain situations where it can be useful to do some filtering
on the client side, but those are micro-optimisations that really only come in very later.</p>
Anyway, now, let’s dig in to the rest of your question and more specifically, let’s try to figure
out what was going wrong. If we run your original query directly in MySQL, we’ll notice that we get
interesting results:</p>
mysql</span>></span> select</span> *</span></span>
    -></span> from</span> uploads</span></span>
    -></span> where</span> af_uid </span>=</span> 101</span></span>
    -></span> order by</span> af_upload_date </span>desc</span></span>
    -></span> limit</span> 5</span>;</span></span>
+</span>--------+--------+--------------+---------------------+</span></span>
| af_uid | af_fid | af_dfilename | af_upload_date      |</span></span>
+</span>--------+--------+--------------+---------------------+</span></span>
|    </span>101</span> |     </span>10</span> | mysql.jpg    | </span>2015</span>-</span>08</span>-</span>16 14</span>:</span>46</span>:</span>52</span> |</span></span>
|    </span>101</span> |     </span>10</span> | php.jpg      | </span>2015</span>-</span>08</span>-</span>16 14</span>:</span>46</span>:</span>10</span> |</span></span>
|    </span>101</span> |     </span>10</span> | bar.jpg      | </span>2015</span>-</span>08</span>-</span>16 14</span>:</span>45</span>:</span>36</span> |</span></span>
|    </span>101</span> |     </span>10</span> | foo.jpg      | </span>2015</span>-</span>08</span>-</span>16 14</span>:</span>45</span>:</span>00</span> |</span></span>
|    </span>101</span> |     </span>11</span> | doc.pdf      | </span>2015</span>-</span>08</span>-</span>16 14</span>:</span>44</span>:</span>23</span> |</span></span>
+</span>--------+--------+--------------+---------------------+</span></span>
5</span> rows in set</span> (</span>0</span>.</span>00</span> sec)</span></span></code></pre>
Notice that because of the order in which things appear in the table (due to the order by</code> clause),
the last item actually has af_fid = 11</code>.</p>
Now, if on the client side, we apply extra filtering as you’re doing with if ($row['af_fid']) == 10)</code>, it would only make sense that we end up with only 4 results.</p>
I’m not sure why you saw 3 results, but I’m going to warrant a guess and say that it was due to some
differences in the code or data, compared to what you posted on SO. Conclusion: if you’d done all
your filtering in SQL from the get go, you most probably would’ve never hit this roadblock to start
with.</p>
Feel free to ping me if you have other questions.</p>


Hash anagram challenge, part 2
2015-02-03T00:00:00+00:00
This is part two of my attempt to solve the Hash Anagram challenge offered by TrustPilot. Full
disclosure: These articles are not written in chronological order; I finished the challenge a few
weeks ago. I’m merely documenting my mental process, and trying to get back into proper writing. In
this part, we’ll start writing a small script that crunches the words, and do some rough
benchmarking on it, to see whether our approach actually makes sense or not.</p>
In part 1</a>, we saw that we have a couple thousand words to work with. We also saw that
working from the letters in the seed anagram alone, there were a few quintillion possible
permutations.</p>
So what kind of algorithms could we apply to this problem, to try and solve it? Let’s try to restate
the problem in order to see things more clearly:</p>

Given a list of 1758 words, find a permutation of n</em> words whose md5 hash computes to
4624d200580677270a54ccff86b9610e</code>.</p>
</blockquote>
First, let’s lay down some observations:</p>

There are 18 letters in the seed anagram, hence the number of letters in the target anagram will
be 18 as well;</li>
We don’t know how many words the target anagram contains;</li>
We don’t know how many permutations of the words list are actual anagrams.</li>
</ol>
Technically, we could assume that the target anagram contains nothing more than three words, but I
don’t want to make that assumption.</p>
I see two main algorithmic paths:</p>

Generate all possible word permutations, and filter it down;</li>
Recursively append words to one another until an anagram is found.</li>
</ul>
So how do you check whether a string is an anagram of another string? I’ve found the following
algorithm in Python:</p>
def</span> isAnagram</span>(</span>first</span>,</span> second</span>):</span></span>
    # If the strings are identical, they are anagrams</span></span>
    if</span> first</span> ==</span> second</span>:</span></span>
        return</span> True</span></span>
</span>
    # Remove spaces, and convert `first` into a list</span></span>
    first</span> =</span> first</span>.</span>translate</span>({</span>ord</span>(</span>' '</span>):</span> None</span>})</span></span>
    second</span> =</span> second</span>.</span>translate</span>({</span>ord</span>(</span>' '</span>):</span> None</span>})</span></span>
</span>
    # If they don't have the same length, they can't be anagrams</span></span>
    if</span> len</span>(</span>first</span>)</span> !=</span> len</span>(</span>second</span>):</span></span>
        return</span> False</span></span>
</span>
    else</span>:</span></span>
        first</span> =</span> list</span>(</span>first</span>)</span></span>
</span>
        try</span>:</span></span>
            # Remove all characters found in second</span></span>
            map</span>(</span>first</span>.</span>remove</span>,</span> second</span>)</span></span>
        except</span> ValueError</span>:</span></span>
            return</span> False</span></span>
        else</span>:</span></span>
            return</span> True</span></span></code></pre>
And an equivalent implementation in C:</p>
// Return a copy of the letters a string (without spaces)</span></span>
char *</span> lttrdup</span>(</span>char const *</span> str</span>) {</span></span>
  // Technically, we're wasting a few bytes here, but that's OK.</span></span>
  char</span> *</span> copy </span>=</span> malloc</span>(</span>strlen</span>(</span>str</span>)</span> +</span> 1</span>);</span></span>
  char</span> *</span> tmp </span>=</span> copy</span>;</span></span>
</span>
  do</span> {</span></span>
    if</span> (</span>*</span>str </span>!=</span> ' '</span>)</span></span>
      *</span>tmp</span>++ = *</span>str</span>;</span></span>
  }</span> while</span> (</span>*</span>str</span>++ !=</span> '</span>\0</span>'</span>);</span></span>
</span>
  return</span> copy</span>;</span></span>
}</span></span>
</span>
bool</span> is_anagram</span>(</span>char const *</span> first</span>,</span> char const *</span> second</span>) {</span></span>
  // If the strings are equivalent, they are anagrams</span></span>
  if</span> (</span>strcmp</span>(</span>first</span>,</span> second</span>)</span> ==</span> 0</span>)</span></span>
    return</span> true</span>;</span></span>
</span>
  else</span> {</span></span>
    // Remove the spaces from our strings</span></span>
    char</span> *</span> cpy_first </span>=</span> lttrdup</span>(</span>first</span>);</span></span>
    char</span> *</span> cpy_second </span>=</span> lttrdup</span>(</span>second</span>);</span></span>
</span>
    // tmp_second is an iterator over cpy_second</span></span>
    char</span> *</span> tmp_second </span>=</span> cpy_second</span>;</span></span>
</span>
    // If both strings don't have the same length, they can't be anagrams</span></span>
    if</span> (</span>strlen</span>(</span>cpy_first</span>)</span> ==</span> strlen</span>(</span>cpy_second</span>)) {</span></span>
</span>
      // Iterate over cpy_second until the end is reached</span></span>
      while</span> (</span>*</span>tmp_second </span>!=</span> '</span>\0</span>'</span>) {</span></span>
        // find the character pointed by tmp_second in cpy_first</span></span>
        char</span> *</span> match </span>=</span> strchr</span>(</span>cpy_first</span>,</span> *</span>tmp_second</span>);</span></span>
</span>
        // if a match is found, overwrite it, and move on the to the</span></span>
        // next character</span></span>
        if</span> (</span>match </span>!=</span> NULL</span>) {</span></span>
          *</span>match </span>=</span> '.'</span>;</span></span>
          tmp_second</span>++</span>;</span></span>
        }</span></span>
</span>
        // no match found, skip to cleanup</span></span>
        else</span></span>
          break</span>;</span></span>
      }</span></span>
</span>
    }</span></span>
</span>
    // if tmp_second doesn't point to the end of the string, we found a</span></span>
    // character that doesn't exist in first, hence, no anagram</span></span>
    bool</span> result </span>= *</span>tmp_second </span>==</span> '</span>\0</span>'</span>;</span></span>
</span>
    // cleanup</span></span>
    free</span>(</span>cpy_first</span>);</span></span>
    free</span>(</span>cpy_second</span>);</span></span>
</span>
    return</span> result</span>;</span></span>
  }</span></span>
}</span></span></code></pre>
The main issue I see with both implementations is that we actually need to do allocations in order
to verify that two strings are anagrams of each other. Granted, the allocations in C are way smaller
and more manageable than the Python ones, but still, not great. If you know of an algorithm that
doesn’t require allocations, I’d be happy to hear about it.</p>
Talking about allocations, what could we do to reduce their numbers? At the moment, the only check
we do before undergoing any allocation, both in Python and C, is checking whether both strings are
the same. How could we cut that down? This is my train of thought:</p>
Characters are encoded into their ASCII value, and for lowercase letters in the alphabet, their
value will range between 97 and 122. What happens if we add the values of our seed anagram (letters
only, no spaces)?</p>
anagram</span> =</span> "poultry outwits ants"</span>.</span>translate</span>(</span>None</span>,</span> ' '</span>)</span></span>
sum</span>(</span>map</span>(</span>ord</span>,</span> anagram</span>))</span></span>
=></span> 2036</span></span></code></pre>
Regardless of the order of the letters, an anagram of “poultry outwits ants” will always have the
same order point sum. The reverse, however, isn’t true: a string that has an order point sum value
of 2036 isn’t necessarily an anagram of “poultry outwits ants”. Let’s throw that into our code:</p>
def</span> letterCount</span>(</span>word</span>):</span></span>
    return</span> sum</span>(</span>map</span>(</span>lambda</span> x</span>:</span> 1</span> if</span> x</span> !=</span> ' '</span> else</span> 0</span>,</span> word</span>))</span></span>
</span>
def</span> sumLetters</span>(</span>word</span>):</span></span>
    return</span> sum</span>(</span>map</span>(</span>lambda</span> x</span>:</span> ord</span>(</span>x</span>)</span> if</span> x</span> !=</span> ' '</span> else</span> 0</span>,</span> word</span>))</span></span>
</span>
def</span> isPlausibleAnagram</span>(</span>first</span>,</span> second</span>):</span></span>
    return</span> letterCount</span>(</span>first</span>)</span> ==</span> letterCount</span>(</span>second</span>)</span> and</span> \</span></span>
        sumLetters</span>(</span>first</span>)</span> ==</span> sumLetters</span>(</span>second</span>)</span></span></code></pre>
Excellent. Let’s try to do the same in C:</p>
bool</span> is_plausible_anagram</span>(</span>char const *</span> first</span>,</span> char const *</span> second</span>) {</span></span>
  unsigned int</span> first_length </span>=</span> 0</span>;</span></span>
  unsigned int</span> second_length </span>=</span> 0</span>;</span></span>
  unsigned int</span> first_ord </span>=</span> 0</span>;</span></span>
  unsigned int</span> second_ord </span>=</span> 0</span>;</span></span>
</span>
  do</span> {</span></span>
    if</span> (</span>*</span>first </span>!=</span> ' '</span>) {</span></span>
      first_ord </span>+= *</span>first</span>;</span></span>
      first_length</span>++</span>;</span></span>
    }</span></span>
  }</span> while</span> (</span>*</span>first</span>++ !=</span> '</span>\0</span>'</span>);</span></span>
</span>
  do</span> {</span></span>
    if</span> (</span>*</span>second </span>!=</span> ' '</span>) {</span></span>
      second_ord </span>+= *</span>second</span>;</span></span>
      second_length</span>++</span>;</span></span>
    }</span></span>
  }</span> while</span> (</span>*</span>second</span>++ !=</span> '</span>\0</span>'</span>);</span></span>
</span>
  return</span> first_length </span>==</span> second_length </span>&&</span> first_ord </span>==</span> second_ord</span>;</span></span>
}</span></span></code></pre>
I decided to write the C version as a single function, so as to only have to loop over the two
strings once. The Python version uses a more functional approach.</p>
Great, so we now have a way to approximate whether a string might be an anagram without doing any
allocations, and a way to know whether a string really is an anagram of another string. Let’s put
together a quick script that gives us all the permutations and starts pumping out anagrams, but
first, let’s use our newfound quicktester to spare some CPU cycles:</p>
def</span> isAnagram</span>(</span>first</span>,</span> second</span>):</span></span>
    if not</span> isPlausibleAnagram</span>(</span>first</span>,</span> second</span>):</span></span>
        return</span> False</span></span>
</span>
    # If the strings are identical, they are anagrams</span></span>
    if</span> first</span> ==</span> second</span>:</span></span>
        return</span> True</span></span>
</span>
    first</span> =</span> first</span>.</span>translate</span>({</span>ord</span>(</span>' '</span>):</span> None</span>})</span></span>
    second</span> =</span> second</span>.</span>translate</span>({</span>ord</span>(</span>' '</span>):</span> None</span>})</span></span>
</span>
    # If they don't have the same length, they can't be anagrams</span></span>
    if</span> len</span>(</span>first</span>)</span> !=</span> len</span>(</span>second</span>):</span></span>
        return</span> False</span></span>
</span>
    else</span>:</span></span>
        first</span> =</span> list</span>(</span>first</span>)</span></span>
</span>
        try</span>:</span></span>
            # Remove all characters found in second</span></span>
            map</span>(</span>first</span>.</span>remove</span>,</span> second</span>)</span></span>
        except</span> ValueError</span>:</span></span>
            return</span> False</span></span>
        else</span>:</span></span>
            return</span> True</span></span></code></pre>
And now for the main function:</p>
def</span> main</span>():</span></span>
    words_url</span> =</span> 'http://followthewhiterabbit.trustpilot.com/cs/wordlist'</span></span>
    anagram</span> =</span> 'poultryoutwitsants'</span></span>
</span>
    words</span> =</span> getEnglishWords</span>(</span>words_url</span>)</span></span>
    words</span> =</span> filterWords</span>(</span>words</span>,</span> anagram</span>)</span></span>
</span>
    length</span> =</span> 0</span></span>
    while</span> True</span>:</span></span>
        length</span> +=</span> 1</span></span>
        print</span>(</span>"Testing permutations of </span>%d</span> words"</span> %</span> length</span>)</span></span>
        generator</span> =</span> itertools</span>.</span>permutations</span>(</span>words</span>,</span> length</span>)</span></span>
</span>
        for</span> permutation</span> in</span> generator</span>:</span></span>
            permutation</span> =</span> ' '</span>.</span>join</span>(</span>permutation</span>)</span></span>
            if</span> isAnagram</span>(</span>permutation</span>,</span> anagram</span>):</span></span>
                print</span>(</span>"--> Found anagram:"</span>,</span> permutation</span>)</span></span></code></pre>
I let this run for a few minutes, and as luck would have it, it actually spat out the answer:</p>
python permutations.py</span></span>
Splitting file into list()</span></span>
Removing words that contain letters not found in anagram</span></span>
Original set of words length: 99174</span></span>
Filtered set of words length: 1658</span></span>
Testing permutations of 1 words</span></span>
Testing permutations of 2 words</span></span>
Testing permutations of 3 words</span></span>
--> Found anagram: want spit tortuously</span></span>
--> Found anagram: want pursuit tolstoy</span></span>
--> Found anagram: want tortuously spit</span></span>
--> Found anagram: want tortuously pits</span></span>
--> Found anagram: want tortuously tips</span></span>
--> Found anagram: want pits tortuously</span></span>
--> Found anagram: want tolstoy pursuit</span></span>
--> Found anagram: want lousy outstript</span></span>
--> Found anagram: want yous trustpilot</span></span>
--> Found anagram: want trustpilot yous</span></span></code></pre>
Wait, trustpilot</code>? Would the answer just be “trustpilot wants you</em>”? Nevermind, I haven’t tested
any md5 hashes or anything, so I won’t cheat. The major issue with using itertools.permutations</code> is
that it’s pretty difficult to run on a cluster, and looking at the speed, it’s not going to complete
quickly. What I mean by speed is that a quick benchmark shows that the Python script running on my
netbook is capable of rougly 50k permutations per second. Going back to the number crunching we did
in part 1, how many different permutations are there for n</em> ranging from 1 to 5?</p>
1658! / (1658-1)! + 1658! / (1658-2)! + 1658! / (1658-3)! + \</span></span>
    1658! / (1658-4)! + 1658! / (1658-5)!</span></span>
=> 12 461 304 888 660 100</span></span></code></pre>
I don’t even need to calculate how long that would take to know it would be a very, very long time
(hint: they should’ve started back in the Copper Age</a>). Now obviously, I wouldn’t
need to test every single</em> permutation; especially if I check the md5 hash every time I find an
anagram.</p>
So how can we split the work across multiple cores? I know just the library for the job:
ZeroMQ</a>. ZeroMQ (also written ØMQ) is a great tool to have on your belt. It doesn’t
require much, other than a good ol’ paradigm shift (POP!).</p>
</p>
ZeroMQ makes multithreading and multiprocessing easy. Well, it doesn’t, it just makes communicating
between multiple threads, processes or servers a piece of cake. The PyZMQ bindings</a> are
excellent, and they make everything an absolute pleasure to work with. ZeroMQ takes care of managing
the sockets, establishing or receiving connections, polling and whatnot. Instead of working with
dumb sockets, ZeroMQ offers pre-defined socket types that behave in certain ways (request/response,
router, publish/subscribe, etc.).</p>
Here’s the architecture we’ll deploy, simple and concise:</p>

The server will take care of downloading and reducing the wordlist;</li>
When a client asks for a word, the server will sequentially go through its words list, and send
the next word to a client;</li>
Upon exhausting all possibilities for a given word, the client will send back all the anagrams it
found, and request a new word.</li>
</ul>
Later on, we’ll teach clients to also switch to md5 computing tasks if there are any. This will
allow us to not spend the next 8000 years computing, but also completing the challenge.</p>
ZeroMQ gives us a socket we can talk on between the clients and the server, but what dialect should
we employ? These days, I usually default to JSON; it’s very well supported and can encode nearly all
the types of data one usually needs. It compresses fairly well, and isn’t too verbose, yet remains
human-readable. And hey look! PyZMQ offers convenience functions to read/write in JSON directly!</p>
The other question we should probably answer is “What kind of sockets are we going to use?” For the
sake of simplicity, we’re going to use one of ZeroMQ’s simplest socket types: the Request-Response
combo. Let’s take a moment to define the grammar of our dialect:</p>
--> {'type': 'initialise'}</span></span>
<-- {'seed': 'poultryoutwitsants', 'words': ['a', 'abaci', 'aback', ... ]}</span></span>
</span>
--> {'type': 'next_word'}</span></span>
<-- {'word': 'truants'}</span></span>
</span>
--> {'type': 'found_anagrams', 'result': ['truants outwit slopy', ...]}</span></span>
<-- {}</span></span></code></pre>
Let’s start by creating our server. Most of the stuff will remain the same, we’re just going to add
a small class that keeps track of the current word, and stuff like that:</p>
class</span> WordsList</span>(</span>object</span>):</span></span>
    """I download a list of words, clean it up and iterate through it."""</span></span>
</span>
    def</span> __init__</span>(</span>self</span>,</span> filter</span>,</span> url</span>):</span></span>
        self</span>.</span>_words</span> =</span> getEnglishWords</span>(</span>url</span>)</span></span>
        self</span>.</span>_words</span> =</span> filterWords</span>(</span>self</span>.</span>_words</span>,</span> filter</span>)</span></span>
    # A copy for future reference</span></span>
    self</span>.</span>_all_words</span> =</span> list</span>(</span>self</span>.</span>_words</span>)</span></span>
</span>
    def</span> nextWord</span>(</span>self</span>):</span></span>
        """I give the next word in the words list, or None if the end has been</span></span>
        reached"""</span></span>
</span>
        try</span>:</span></span>
            return</span> self</span>.</span>_words</span>.</span>pop</span>()</span></span>
        except</span> IndexError</span>:</span></span>
            return</span> None</span></span>
</span>
    def</span> getWords</span>(</span>self</span>):</span></span>
        """I give access to the whole words list."""</span></span>
        return</span> self</span>.</span>_all_words</span></span>
</span>
    def</span> endReached</span>(</span>self</span>):</span></span>
        """I'm a helper function that tells you whether the end of the list has</span></span>
        been reached or not"""</span></span>
        return</span> len</span>(</span>self</span>.</span>_words</span>)</span> ==</span> 0</span></span></code></pre>
And this is again where I’m going to call it a night ;) I’ll finish writing this article tomorrow or
something. Thanks for reading, anyway.</p>


Hash anagram challenge, part 1
2015-02-02T00:00:00+00:00
As I mentioned previously, I’ve been working on a technical challenge I found a few days ago. Just
to be clear: this is a low-priority challenge for me. I don’t expect anything out of this, and I
don’t consider it to be anything more than yet another toy-project to keep my mind busy whenever I
have a moment to spare.</p>
The challenge</a> is as follows:</p>

What is the anagram of “poultry outwits ants” whose md5 hash computes down to
4624d200580677270a54ccff86b9610e</code>?</p>
</blockquote>
Sounds simple enough, right? That’s what I thought as well. Let’s start by doing some math:</p>
There are 18 letters in the seed anagram, not including spaces. Let’s leave the spaces aside for the
moment and just focus on those 18 letters. How many permutations of letters are there? Well, that’s
fairly easy, let’s head up to WolframAlpha</a> and calculate some
factorials</a>:</p>
18!</span></span>
=> 6 402 373 705 728 000</span></span></code></pre>
Just shy of 6.5 quadrillion</a> possibilities? Not bad. Now, let’s see what throwing in
those spaces changes…</p>
21!</span></span>
=> 51 090 942 171 709 440 000</span></span>
</span>
23!</span></span>
=> 25 852 016 738 884 976 640 000</span></span></code></pre>
Between 51 quintillion</a> possibilities and 25 sixtillion possibilities</a>,
assuming there are respectively 3 to 5 words in the actual anagram. Wow. We now know how many
possible permutations there are. These are purely permutations (e.g.: theoretical anagrams), and not
actually matched to real, existing words. That’s another step. Let’s just go with these numbers and
see how long it would take to calculate all of these.</p>
How fast is my computer? I’m writing this on a small, cheap netbook that runs off battery power. I’m
not expecting much out of it. Let’s see what John the ripper</a> thinks of my CPU.</p>
$ john -test | grep -A md5</span></span>
Benchmarking: md5crypt [MD5 32/64 X2]... DONE</span></span>
Raw: 5383 c/s real, 5427 c/s virtual</span></span></code></pre>
I’m taking md5crypt as a low baseline, as I’m guesstimating it to be more computationally intensive
than just calculating the md5 sum of small strings. Let’s assume I get a thousand boxes like mine,
slightly more powerful (four cores instead of two):</p>
5000 hashes/s * 1 000 000 * 4 cores</span></span>
=> 20 000 000 000</span></span></code></pre>
20 billion hashes per second, that’s pretty good, and in our era of cloud computing, surely not that
impossible to achieve. How long would it take me to crack the challenge, by simple, honest to
goodness bruteforce?</p>
21! / (5000 * 4 * 1 000 000 per second)</span></span>
=> 709 596 hours</span></span>
=> 29 567 days</span></span></code></pre>
So, assuming perfect workload distribution, and assuming no overhead from the software, it would
take around 80 years</a> to calculate the md5 hash of every single possible anagram.
Now, I realise those are very, very low values; maybe more worthy of vapour that cloud computing.
According to Andrew Dunham</a>, it is possible to get up to 2.3 billion md5 hashes per second
per Amazon EC2 GPU cluster. So assuming an hourly rate of USD 0.7, it would only cost under USD 5
million</a> to calculate everything, but it would still take nearly as
long</a> as to put a human on this planet. Note that those numbers are for 1000 GPU
clusters; The reason I didn’t go for the million-figure in this estimate is because I somehow
strongly doubt the fact that Amazon has a spare million GPU clusters lying around.</p>
I love a good challenge as much as the next guy, but I don’t have that kind of time or money to
throw at just something to keep my brain idling. So where do we go from now? Let’s look at the
challenge again. Hey, they gave us a list of words</a>! That must</em> be useful, right? Of
course it is!</p>
How many words did they give us?</p>
$</span> wc -l wordlist</span></span>
99174</span> wordlist</span></span></code></pre>
Wait, how many unique words did they give us?</p>
$</span> sort -u wordlist</span> |</span> wc</span> -l</span></span>
96316</span></span></code></pre>
OK, so let’s remove any word that contains a letter that doesn’t exist in our seed anagram: poultry
outwits ants</em>. Time to let go of bash and friends, and call in the big guns: Python.</p>
#!/usr/bin/env python</span></span>
</span>
from</span> __future__</span> import</span> (</span>absolute_import</span>,</span> print_function</span>,</span></span>
                        unicode_literals</span>)</span></span>
</span>
import</span> os</span>.</span>path</span></span>
import</span> urllib2</span></span>
</span>
</span>
"""</span></span>
Unoptimised file downloader with minimal cache management. It expects a</span></span>
newline-delimited file and removes any line that contains letters not found in</span></span>
an arbritrary string</span></span>
"""</span></span>
</span>
</span>
def</span> fileFromHTTP</span>(</span>url</span>,</span> filename</span>):</span></span>
    """I download a file and save it to the provided filename"""</span></span>
</span>
    print</span>(</span>"Downloading file:"</span>,</span> url</span>)</span></span>
    file</span> =</span> urllib2</span>.</span>urlopen</span>(</span>url</span>).</span>read</span>()</span></span>
</span>
    with</span> open</span>(</span>filename</span>,</span> 'wb'</span>)</span> as</span> fp</span>:</span></span>
        fp</span>.</span>write</span>(</span>file</span>)</span></span>
</span>
</span>
def</span> fileToList</span>(</span>file</span>):</span></span>
    """I know how split a file into a list(). Yay me."""</span></span>
</span>
    print</span>(</span>"Splitting file into list()"</span>)</span></span>
    lines</span> =</span> [</span>x</span>.</span>decode</span>(</span>'utf-8'</span>)</span> for</span> x</span> in</span> file</span>.</span>splitlines</span>()]</span></span>
    return</span> lines</span></span>
</span>
</span>
def</span> getEnglishWords</span>(</span>url</span>):</span></span>
    """I check if the cache is warm, and download the file if it can't be found</span></span>
    locally"""</span></span>
</span>
    filename</span> =</span> os</span>.</span>path</span>.</span>basename</span>(</span>url</span>)</span></span>
</span>
    if not</span> os</span>.</span>path</span>.</span>isfile</span>(</span>filename</span>):</span></span>
        fileFromHTTP</span>(</span>url</span>,</span> filename</span>)</span></span>
</span>
    with</span> open</span>(</span>filename</span>,</span> 'rb'</span>)</span> as</span> fp</span>:</span></span>
        file</span> =</span> fp</span>.</span>read</span>()</span></span>
</span>
    return</span> fileToList</span>(</span>file</span>)</span></span>
</span>
</span>
def</span> filterWords</span>(</span>haystack</span>,</span> needles</span>):</span></span>
    """I take a a list of words, and remove any that can't be found in the</span></span>
    needles string"""</span></span>
</span>
    print</span>(</span>"Removing words that contain letters not found in anagram"</span>)</span></span>
    print</span>(</span>"Original set of words length:"</span>,</span> len</span>(</span>haystack</span>))</span></span>
</span>
    # The list(set()) operation is done to remove dupes</span></span>
    haystack</span> =</span> list</span>(</span>set</span>(</span>haystack</span>))</span></span>
</span>
    for</span> word</span> in</span> list</span>(</span>haystack</span>):</span></span>
        if not</span> existsInPool</span>(</span>needles</span>,</span> word</span>):</span></span>
            haystack</span>.</span>remove</span>(</span>word</span>)</span></span>
</span>
    print</span>(</span>"Filtered set of words length:"</span>,</span> len</span>(</span>haystack</span>))</span></span>
    return</span> haystack</span></span>
</span>
</span>
def</span> existsInPool</span>(</span>pool</span>,</span> word</span>):</span></span>
    """I check whether `word` could possibly exist in `pool`"""</span></span>
    pool</span> =</span> list</span>(</span>pool</span>)</span></span>
</span>
    try</span>:</span></span>
        for</span> c</span> in</span> word</span>:</span></span>
            pool</span>.</span>remove</span>(</span>c</span>)</span></span>
    except</span> ValueError</span>:</span></span>
        return</span> False</span></span>
</span>
    return</span> True</span></span>
</span>
</span>
def</span> main</span>():</span></span>
    words_url</span> =</span> 'http://followthewhiterabbit.trustpilot.com/cs/wordlist'</span></span>
    anagram</span> =</span> 'poultryoutwitsants'</span></span>
</span>
    words</span> =</span> getEnglishWords</span>(</span>words_url</span>)</span></span>
    words</span> =</span> filterWords</span>(</span>words</span>,</span> anagram</span>)</span></span>
</span>
</span>
if</span> __name__</span> ==</span> '__main__'</span>:</span></span>
    main</span>()</span></span></code></pre>
Let’s give it a run and see how many words we managed to squash out:</p>
$</span> python foo.py</span></span>
Splitting</span> file into list</span>()</span></span>
Removing</span> words that contain letters not found in anagram</span></span>
Original</span> set of words length:</span> 99174</span></span>
Filtered</span> set of words length:</span> 1658</span></span></code></pre>
OK, we managed to remove 98.32% of the words list provided by Trustpilot. This definitely reduces
the amount of calculations we’re going to have to perform, but it doesn’t really make things that
much easier. Let’s look at why.</p>
The usual way to calculate the number of permutations will again be based around factorials, as we
used previously. As before, let’s assume the target anagram is made of 3 words, just like the seed
anagram. What would be the formula to calculate that? Well actually, the Python
documentation</a> gives it right to us:</p>
# for n the pool length, and r the number of permutations:</span></span>
n! / (n-r)!</span></span></code></pre>
So applied to our problem:</p>
1658! / (1658 - 3)!</span></span>
=> 4 549 538 736</span></span>
</span>
# 18 bytes per anagram, 3 bytes for spaces, 32 bytes for the md5 hash</span></span>
4549538736 * (18 bytes + 3 bytes + 32 bytes)</span></span>
=> 241 125 553 008</span></span></code></pre>
I guess it would be possible to load up a Cassandra cluster with 240GB of data and use Apache Spark
to carve through it, but it doesn’t really seem like the right way to do it.</p>
In the next article (part 2</a>), we’ll write a quick python script that starts crunching
through the list of words and find a way to discard impossible permutations.</p>


RPM versioning with Jenkins
2014-02-01T00:00:00+00:00
I’m currently deploying Jenkins at work, in order to simplify the build process. At the moment, we
have a fairly complicated process that involves a couple dozen steps, which, even though each step
is fairly braindead, does require a good amount of focus and planning in order to get everything
just right.</p>
Jenkins</a> provides a build number for all the different projects you might configure. Each build
number is unique in a given project; this ensures that you can refer to specific builds without too
much of a hassle. That’s great, but knowing that I’m trying to achieve fully automated RPM
packaging, it doesn’t help me much.</p>
The versioning system in our company (defined by moi, pretty much), is heavily inspired by the usual
RPM rules: release numbers are reset to 1 whenever the version number changes. There’s no real way,
as far as I know, to use Jenkins’ build number for this, as there is no programmatic way to reset
the build number in Jenkins.</p>
Multi-instance exclusive lock in Python</h1>
The first issue I tried to tackle was the fact that the Jenkins server will run multiple builds at
the same time, possibly for the same project. What this means is that simply having stuff in a file
isn’t really sensible due to concurrency issues. I know that portalocker</a> roughly does the same
as what I tried to do, but not completely. portalocker provides a lock to a file; my implementation
provides a transaction-kinda lock, regardless of whether you want to access files or databases or
whatever.</p>
Without further ado, here’s the class:</p>
import</span> os</span>,</span> time</span></span>
</span>
class</span> Lock</span>(</span>object</span>):</span></span>
    """</span></span>
    Manage a system-wide lock used to inform other processes they should wait.</span></span>
    The lockfile argument is the token that should be shared amongst all the</span></span>
    processes wanting to acquire exclusive locks to the same resource.</span></span>
    """</span></span>
</span>
    polling</span> =</span> 0.1</span></span>
</span>
    def</span> __init__</span>(</span>self</span>,</span> lockfile</span>):</span></span>
        self</span>.</span>_lockfile</span> =</span> '</span>%s</span>.lock'</span> %</span> lockfile</span></span>
        # I only target linux systems... Sorry.</span></span>
        self</span>.</span>_path</span> =</span> os</span>.</span>path</span>.</span>join</span>(</span>'/tmp'</span>,</span> self</span>.</span>_lockfile</span>)</span></span>
        self</span>.</span>locked</span> =</span> False</span></span>
</span>
    def</span> __enter__</span>(</span>self</span>):</span></span>
        self</span>.</span>acquire</span>()</span></span>
        return</span> self</span></span>
</span>
    def</span> __exit__</span>(</span>self</span>,</span> _</span>,</span> __</span>,</span> ___</span>):</span></span>
        self</span>.</span>release</span>()</span></span>
</span>
    def</span> acquire</span>(</span>self</span>):</span></span>
        """</span></span>
        Try to acquire a lock. This is a blocking call, until the lock is</span></span>
        acquired. If you want a non-blocking call, try nb_acquire().</span></span>
        """</span></span>
        if</span> self</span>.</span>locked</span>:</span></span>
            """</span> TODO</span>: Raise """</span></span>
            pass</span></span>
</span>
        while not</span> self</span>.</span>_acquire</span>():</span></span>
            time</span>.</span>sleep</span>(</span>self</span>.</span>polling</span>)</span></span>
</span>
        self</span>.</span>locked</span> =</span> True</span></span>
</span>
    def</span> nb_acquire</span>(</span>self</span>):</span></span>
        if</span> self</span>.</span>locked</span>:</span></span>
            """</span> TODO</span>: Raise """</span></span>
            pass</span></span>
</span>
        self</span>.</span>locked</span> =</span> self</span>.</span>_acquire</span>()</span></span>
        return</span> self</span>.</span>locked</span></span>
</span>
    def</span> _acquire</span>(</span>self</span>):</span></span>
        try</span>:</span></span>
            os</span>.</span>mkdir</span>(</span>self</span>.</span>_path</span>)</span></span>
        except</span> OSError</span>:</span></span>
            return</span> False</span></span>
        return</span> True</span></span>
</span>
    def</span> release</span>(</span>self</span>):</span></span>
        if not</span> self</span>.</span>locked</span>:</span></span>
            """</span> TODO</span>: Raise """</span></span>
            pass</span></span>
</span>
        os</span>.</span>rmdir</span>(</span>self</span>.</span>_path</span>)</span></span>
</span>
        self</span>.</span>locked</span> =</span> False</span></span></code></pre>
There’s a couple of TODOs, and I might push this stuff to my github account at some point or
another, but it gives a general idea on what I tried to do. The jist of it is that I use mkdir as an
atomic locking operation. What you lock against is basically just an identifier, which means that
whether you want to read/write to files, access a database, fiddle your toes doesn’t realy matter.
If the lock acquisition fails, the acquire()</code> method will block until it is able to acquire the
lock. If you don’t want it to block, there’s always nb_acquire()</code> which will just return a boolean
indicating the lock status.</p>
How to use it</h2>
Here’s how it could be used:</p>
lock_name</span> =</span> 'my-lock'</span></span>
</span>
# Primary usage</span></span>
with</span> Lock</span>(</span>lock_name</span>):</span></span>
    print</span> "We should be locked!"</span></span></code></pre>
And that’s really just it. The locking magic happens in the with</code> statement. As soon you enter the
with</code> statement, you are guaranteed to be the sole executor of whatever code you have running. As
soon as you leave the scope of the with</code> statement, the lock is released. Below are a few other
scenarios of how you could use the above class:</p>
lock_name</span> =</span> 'my-lock'</span></span>
</span>
# Another kind of usage</span></span>
lock</span> =</span> Lock</span>(</span>lock_name</span>)</span></span>
lock</span>.</span>acquire</span>()</span></span>
print</span> "We should be locked!"</span></span>
</span>
# If you do not call the following function, you're going to have bugs.</span></span>
lock</span>.</span>release</span>()</span></span>
</span>
# Yet another kind of usage (non-blocking, this time)</span></span>
if</span> lock</span>.</span>nb_acquire</span>():</span></span>
    print</span> "We should be locked!"</span></span>
    lock</span>.</span>release</span>()</span></span>
else</span>:</span></span>
    print</span> "We're most definitely not locked!"</span></span></code></pre>
So, now we can execute a block of code without having to worry about other instances of the script
mucking about our stuff. Let’s work on the RPM versioning now.</p>
Efficient RPM release numbers</h1>
RPM versioning works like this: 1.2.3-4</code> where 1.2.3</code> is the version number, and 4</code> is the
release. For a more in-depth overview of RPM versioning, check out Fedora’s wiki, or something. I
usually put the version number of my software in a file somewhere, so getting that isn’t usually a
problem. However, the release part of the RPM version is a bit more tricky. The way we use it is
that every time we release something to the QA team (or directly to production if it’s a very urgent
bugfix), we increment the release number if the version number hasn’t changed. This is the case, for
example, if you have to fix something in the packaging process, or if you are applying successive
fixes to the release branch (according to nvie’s branching model</a>).</p>
This also means that as soon as the version number changes, the release number should be reset to 1.
I wrote the following tool, which uses the Lock</code> class above, to keep track of all the different
releases that have been built so far, based on the package name and version number:</p>
import</span> argparse</span>,</span> json</span></span>
from</span> lock</span> import</span> Lock</span></span>
from</span> os</span>.</span>path</span> import</span> expanduser</span>,</span> join</span></span>
</span>
class</span> RPMReleases</span>(</span>object</span>):</span></span>
</span>
    def</span> __init__</span>(</span>self</span>):</span></span>
        self</span>.</span>_path</span> =</span> join</span>(</span>expanduser</span>(</span>'~'</span>),</span> 'iv-rpm-releases'</span>)</span></span>
        self</span>.</span>_releases</span> =</span> {}</span></span>
</span>
    def</span> __enter__</span>(</span>self</span>):</span></span>
        self</span>.</span>load</span>()</span></span>
        return</span> self</span></span>
</span>
    def</span> __exit__</span>(</span>self</span>,</span> _</span>,</span> __</span>,</span> ___</span>):</span></span>
        self</span>.</span>dump</span>()</span></span>
</span>
    def</span> load</span>(</span>self</span>):</span></span>
        try</span>:</span></span>
            with</span> open</span>(</span>self</span>.</span>_path</span>,</span> 'r'</span>)</span> as</span> f</span>:</span></span>
                self</span>.</span>_releases</span> =</span> json</span>.</span>load</span>(</span>f</span>)</span></span>
        except</span> IOError</span>:</span></span>
            pass</span></span>
</span>
    def</span> dump</span>(</span>self</span>):</span></span>
        with</span> open</span>(</span>self</span>.</span>_path</span>,</span> 'w+'</span>)</span> as</span> f</span>:</span></span>
            json</span>.</span>dump</span>(</span>self</span>.</span>_releases</span>,</span> f</span>)</span></span>
</span>
    def</span> get_next_release</span>(</span>self</span>,</span> package</span>,</span> version</span>,</span> increment</span> =</span> True</span>):</span></span>
        if</span> package</span> not in</span> self</span>.</span>_releases</span>:</span></span>
            self</span>.</span>_releases</span>[</span>package</span>]</span> =</span> {}</span></span>
</span>
        if</span> version</span> not in</span> self</span>.</span>_releases</span>[</span>package</span>]:</span></span>
            self</span>.</span>_releases</span>[</span>package</span>][</span>version</span>]</span> =</span> 0</span></span>
</span>
        if</span> increment</span>:</span></span>
            self</span>.</span>_releases</span>[</span>package</span>][</span>version</span>]</span> +=</span> 1</span></span>
</span>
        return</span> self</span>.</span>_releases</span>[</span>package</span>][</span>version</span>]</span></span>
</span>
    def</span> set_release</span>(</span>self</span>,</span> package</span>,</span> version</span>,</span> release</span>):</span></span>
        if</span> package</span> not in</span> self</span>.</span>_releases</span>:</span></span>
            self</span>.</span>_releases</span>[</span>package</span>]</span> =</span> {}</span></span>
</span>
        self</span>.</span>_releases</span>[</span>package</span>][</span>version</span>]</span> =</span> release</span></span>
</span>
if</span> __name__</span> ==</span> '__main__'</span>:</span></span>
    parser</span> =</span> argparse</span>.</span>ArgumentParser</span>()</span></span>
    parser</span>.</span>add_argument</span>(</span>"package"</span>,</span></span>
        help</span> =</span> "The software package for which you want a release number"</span>)</span></span>
    parser</span>.</span>add_argument</span>(</span>"version"</span>,</span></span>
        help</span> =</span> "The version of the package for which you want a release number"</span>)</span></span>
</span>
    parser</span>.</span>add_argument</span>(</span>"--current"</span>,</span> action</span> =</span> "store_true"</span>,</span></span>
        help</span> =</span> "Don't increment any counters, just print the current value"</span>)</span></span>
    parser</span>.</span>add_argument</span>(</span>"--full"</span>,</span> action</span> =</span> "store_true"</span>,</span></span>
        help</span> =</span> "Print the full package name"</span>)</span></span>
</span>
    args</span> =</span> parser</span>.</span>parse_args</span>()</span></span>
</span>
    with</span> Lock</span>(</span>'iv-rpm-releases'</span>):</span></span>
        with</span> RPMReleases</span>()</span> as</span> releases</span>:</span></span>
            inc</span> =</span> not</span> args</span>.</span>current</span></span>
            release</span> =</span> releases</span>.</span>get_next_release</span>(</span>args</span>.</span>package</span>,</span> args</span>.</span>version</span>,</span> increment</span> =</span> inc</span>)</span></span>
</span>
            if not</span> args</span>.</span>full</span>:</span></span>
                print</span> release</span></span>
</span>
            else</span>:</span></span>
                print</span> '</span>%s</span>-</span>%s</span>-</span>%s</span>'</span> %</span> (</span>args</span>.</span>package</span>,</span> args</span>.</span>version</span>,</span> release</span>)</span></span></code></pre>How to use it</h2>
The tool is used as follows:</p>
$</span> python rpm-release-number.py -h</span></span>
usage:</span> rpm-release-number.py</span> [-h] </span>[</span>--current</span>] [</span>--full</span>]</span> package version</span></span>
</span>
positional</span> arguments:</span></span>
  package</span>     The software package for which you want a release number</span></span>
  version</span>     The version of the package for which you want a release number</span></span>
</span>
optional</span> arguments:</span></span>
  -h,</span> --help  show this help message and exit</span></span>
  --current</span>   Don't increment any counters, just print the current value</span></span>
  --full      Print the full package name</span></span></code></pre>
For example:</p>
$</span> python rpm-release-number.py foo</span> 1.1</span></span>
1</span></span>
$</span> python rpm-release-number.py foo</span> 1.1</span></span>
2</span></span>
$</span> python rpm-release-number.py foo</span> 1.1</span></span>
3</span></span>
$</span> python rpm-release-number.py foo</span> 1.1</span> --current</span></span>
3</span></span>
$</span> python rpm-release-number.py foo</span> 1.1</span> --current</span></span>
3</span></span>
$</span> python rpm-release-number.py foo</span> 1.1</span> --current --full</span></span>
foo-1.1-3</span></span></code></pre>
So, there we have it. A quick tool that keeps track of what release every package is supposed to
have, and stores it in an easy-to-maintain location (a JSON file in the user’s home directory). This
tool is easily usable in Jenkins scripts, but that’ll be for a different blog entry.</p>

Technically Personal - programming

Testing multiple versions of Python in parallel

New project: comparqter

The Date that wasn't

Rust and WASM for form validation

Stack</h2> This post is being written in early-to-mid 2025. And the stack I currently jive with is as follows:</p>

Project structure</h2> Let’s create a directory structure that allows this whole setup to work. One typical approach for this is to have separate crates for the server part, and the actual wasm component. Something like this:</p>

Server side implementation</h2> Let’s add a few libraries to our server’s Cargo.toml</code>:</p>

Getting started with Scala on macOS in 2025

Freebox automatic ruTorrent download

A reply for Tim's StackOverflow question

Hash anagram challenge, part 2

Hash anagram challenge, part 1

RPM versioning with Jenkins