<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:media="http://search.yahoo.com/mrss/"><channel><title><![CDATA[fuzzknuckle.com]]></title><description><![CDATA[a notebook of half-arsed nerd crap]]></description><link>https://blog.fuzzknuckle.com/</link><image><url>https://blog.fuzzknuckle.com/favicon.png</url><title>fuzzknuckle.com</title><link>https://blog.fuzzknuckle.com/</link></image><generator>Ghost 5.54</generator><lastBuildDate>Thu, 19 Mar 2026 14:55:47 GMT</lastBuildDate><atom:link href="https://blog.fuzzknuckle.com/rss/" rel="self" type="application/rss+xml"/><ttl>60</ttl><item><title><![CDATA[Building go on OPNsense from the ports tree]]></title><description><![CDATA[<p>I was trying to install <a href="https://tailscale.com/?ref=blog.fuzzknuckle.com">TailScale</a> on my <a href="https://opnsense.org/?ref=blog.fuzzknuckle.com">OPNsense</a> firewall by following these instructions: <a href="https://tailscale.com/kb/1097/install-opnsense?ref=blog.fuzzknuckle.com">https://tailscale.com/kb/1097/install-opnsense</a></p>
<p>OPNSense is built on FreeBSD and provides its own ports tree so we can build various packages from source.</p>
<p>Unfortunately I was continually getting build failures when running <code>make install</code></p>]]></description><link>https://blog.fuzzknuckle.com/building-go-on-opnsense-from-the-ports-tree/</link><guid isPermaLink="false">66aafde4a1bf2c0001b76303</guid><dc:creator><![CDATA[jeff]]></dc:creator><pubDate>Thu, 01 Aug 2024 03:34:21 GMT</pubDate><content:encoded><![CDATA[<p>I was trying to install <a href="https://tailscale.com/?ref=blog.fuzzknuckle.com">TailScale</a> on my <a href="https://opnsense.org/?ref=blog.fuzzknuckle.com">OPNsense</a> firewall by following these instructions: <a href="https://tailscale.com/kb/1097/install-opnsense?ref=blog.fuzzknuckle.com">https://tailscale.com/kb/1097/install-opnsense</a></p>
<p>OPNSense is built on FreeBSD and provides its own ports tree so we can build various packages from source.</p>
<p>Unfortunately I was continually getting build failures when running <code>make install</code> in the <code>/usr/ports/security/tailscale</code> dir. I kept failing at the point where we were trying to build <a href="https://go.dev/?ref=blog.fuzzknuckle.com">Go</a>, and was failing at this same point every time I tried. Sometimes accompanied by a hard crash of the server.</p>
<pre><code>bootstrap/cmd/compile/internal/ssa
go build bootstrap/cmd/compile/internal/ssa: /usr/local/go14/pkg/tool/freebsd_amd64/6g: signal: killed
bootstrap/cmd/compile/internal/syntax
bootstrap/cmd/internal/gcprog
bootstrap/internal/goversion
bootstrap/internal/race
bootstrap/cmd/link/internal/sym
bootstrap/cmd/link/internal/loadelf
bootstrap/cmd/link/internal/loadmacho
bootstrap/cmd/link/internal/loadpe
bootstrap/cmd/link/internal/loadxcoff
bootstrap/cmd/link/internal/objfile
bootstrap/cmd/link/internal/ld
bootstrap/cmd/link/internal/amd64
bootstrap/cmd/link/internal/arm
bootstrap/cmd/link/internal/arm64
bootstrap/cmd/link/internal/mips
bootstrap/cmd/link/internal/mips64
bootstrap/cmd/link/internal/ppc64
bootstrap/cmd/link/internal/s390x
bootstrap/cmd/link/internal/wasm
bootstrap/cmd/link/internal/x86
bootstrap/cmd/link
go tool dist: FAILED: /usr/local/go14/bin/go install -gcflags=-l -tags=math_big_pure_go compiler_bootstrap -v bootstrap/cmd/...: exit status 1
*** Error code 2

Stop.
make: stopped in /usr/ports/lang/go

===&gt;&gt;&gt; make build failed for lang/go
===&gt;&gt;&gt; Aborting update

===&gt;&gt;&gt; Update for lang/go failed
===&gt;&gt;&gt; Aborting update

===&gt;&gt;&gt; Update for net/wireguard-go failed
===&gt;&gt;&gt; Aborting update</code></pre>
<p>The only reference I could find on google to this error was an <a href="https://forums.freebsd.org/threads/error-code-2-when-building-lang-go-on-amd64.72669/?ref=blog.fuzzknuckle.com">old post</a> on the FreeBSD forums, which had no resolution. (I copied his error log for the example above, because I wasn&apos;t smart enough to grab one when I was building). Eventually I came across an issue on the <a href="https://github.com/go-bootstrap/go-bootstrap/issues/55?ref=blog.fuzzknuckle.com">Go-Bootstrap</a> git repo, which suggested the process was killed due to an out of memory issue. </p>
<figure class="kg-card kg-image-card"><img src="https://blog.fuzzknuckle.com/content/images/2024/08/image.png" class="kg-image" alt loading="lazy" width="2000" height="384" srcset="https://blog.fuzzknuckle.com/content/images/size/w600/2024/08/image.png 600w, https://blog.fuzzknuckle.com/content/images/size/w1000/2024/08/image.png 1000w, https://blog.fuzzknuckle.com/content/images/size/w1600/2024/08/image.png 1600w, https://blog.fuzzknuckle.com/content/images/2024/08/image.png 2138w" sizes="(min-width: 720px) 720px"></figure>
<p>It finally made sense. I run the OPNSense machine in a VM on proxmox, and over time I had tuned it&apos;s specs down to just what it requires for it&apos;s duties as a router / firewall, trying to build software on it was not something I thought I&apos;d be doing on it. I upped its memory from 1G to 4 temporarily and the build proceeded without any further issue.</p>]]></content:encoded></item><item><title><![CDATA[Vulnhub Machines useful for OSCP]]></title><description><![CDATA[These are taken from #vulnhub, hopefully I'll add some more resources as I find them.]]></description><link>https://blog.fuzzknuckle.com/vulnhub-machines-useful-for-oscp/</link><guid isPermaLink="false">64ad387172191c00015744c5</guid><dc:creator><![CDATA[jeff]]></dc:creator><pubDate>Sun, 26 Feb 2017 08:16:50 GMT</pubDate><media:content url="https://blog.fuzzknuckle.com/content/images/2019/02/hackers.jpg" medium="image"/><content:encoded><![CDATA[<!--kg-card-begin: markdown--><img src="https://blog.fuzzknuckle.com/content/images/2019/02/hackers.jpg" alt="Vulnhub Machines useful for OSCP"><p>These are taken from #vulnhub, hopefully I&apos;ll add some more resources as I find them.</p>
<!--kg-card-end: markdown--><hr><!--kg-card-begin: markdown--><ul>
<li>Kioptrix: 2014 <a href="https://www.vulnhub.com/entry/kioptrix-2014-5,62/?ref=blog.fuzzknuckle.com">https://www.vulnhub.com/entry/kioptrix-2014-5,62/</a></li>
<li>FristiLeaks: 1.3 <a href="https://www.vulnhub.com/entry/fristileaks-13,133/?ref=blog.fuzzknuckle.com">https://www.vulnhub.com/entry/fristileaks-13,133/</a></li>
<li>Stapler: 1 <a href="https://www.vulnhub.com/entry/stapler-1,150/?ref=blog.fuzzknuckle.com">https://www.vulnhub.com/entry/stapler-1,150/</a></li>
<li>VulnOS: 2 <a href="https://www.vulnhub.com/entry/vulnos-2,147/?ref=blog.fuzzknuckle.com">https://www.vulnhub.com/entry/vulnos-2,147/</a></li>
<li>SickOs: 1.2 <a href="https://www.vulnhub.com/entry/sickos-12,144/?ref=blog.fuzzknuckle.com">https://www.vulnhub.com/entry/sickos-12,144/</a></li>
<li>Brainpan: 1 <a href="https://www.vulnhub.com/entry/brainpan-1,51/?ref=blog.fuzzknuckle.com">https://www.vulnhub.com/entry/brainpan-1,51/</a></li>
<li>HackLAB: Vulnix <a href="https://www.vulnhub.com/entry/hacklab-vulnix,48/?ref=blog.fuzzknuckle.com">https://www.vulnhub.com/entry/hacklab-vulnix,48/</a></li>
<li>/dev/random: scream <a href="https://www.vulnhub.com/entry/devrandom-scream,47/?ref=blog.fuzzknuckle.com">https://www.vulnhub.com/entry/devrandom-scream,47/</a></li>
<li>pWnOS: 2.0 <a href="https://www.vulnhub.com/entry/pwnos-20-pre-release,34/?ref=blog.fuzzknuckle.com">https://www.vulnhub.com/entry/pwnos-20-pre-release,34/</a></li>
<li>SkyTower: 1 <a href="https://www.vulnhub.com/entry/skytower-1,96/?ref=blog.fuzzknuckle.com">https://www.vulnhub.com/entry/skytower-1,96/</a></li>
</ul>
<!--kg-card-end: markdown-->]]></content:encoded></item><item><title><![CDATA[Django/Python Development Machine Setup]]></title><description><![CDATA[<!--kg-card-begin: markdown--><h4 id="terminal">Terminal</h4>
<p><strong>Terminator</strong> - <a href="http://gnometerminator.blogspot.com.au/p/introduction.html?ref=blog.fuzzknuckle.com">http://gnometerminator.blogspot.com.au/p/introduction.html</a></p>
<p><strong>prezto</strong> - <a href="https://github.com/sorin-ionescu/prezto?ref=blog.fuzzknuckle.com">https://github.com/sorin-ionescu/prezto</a> <em>(makes zshell easy to customise)</em></p>
<p><strong>Homeshick</strong> - <a href="https://github.com/andsens/homeshick?ref=blog.fuzzknuckle.com">https://github.com/andsens/homeshick</a> <em>(synchronize settings between machines)</em></p>
<h4 id="texteditor">Text Editor</h4>
<p><strong>Sublime Text 3</strong> - <a href="http://www.sublimetext.com/3?ref=blog.fuzzknuckle.com">http://www.sublimetext.com/3</a></p>
<p>I also install a bunch</p>]]></description><link>https://blog.fuzzknuckle.com/dev-machine-setup/</link><guid isPermaLink="false">64ad387172191c00015744c2</guid><category><![CDATA[ubuntu]]></category><category><![CDATA[python]]></category><category><![CDATA[django]]></category><category><![CDATA[development]]></category><dc:creator><![CDATA[jeff]]></dc:creator><pubDate>Mon, 19 Oct 2015 22:14:03 GMT</pubDate><media:content url="https://blog.fuzzknuckle.com/content/images/2019/02/Amstrad_CPC_screen_closeup.jpg" medium="image"/><content:encoded><![CDATA[<!--kg-card-begin: markdown--><h4 id="terminal">Terminal</h4>
<img src="https://blog.fuzzknuckle.com/content/images/2019/02/Amstrad_CPC_screen_closeup.jpg" alt="Django/Python Development Machine Setup"><p><strong>Terminator</strong> - <a href="http://gnometerminator.blogspot.com.au/p/introduction.html?ref=blog.fuzzknuckle.com">http://gnometerminator.blogspot.com.au/p/introduction.html</a></p>
<p><strong>prezto</strong> - <a href="https://github.com/sorin-ionescu/prezto?ref=blog.fuzzknuckle.com">https://github.com/sorin-ionescu/prezto</a> <em>(makes zshell easy to customise)</em></p>
<p><strong>Homeshick</strong> - <a href="https://github.com/andsens/homeshick?ref=blog.fuzzknuckle.com">https://github.com/andsens/homeshick</a> <em>(synchronize settings between machines)</em></p>
<h4 id="texteditor">Text Editor</h4>
<p><strong>Sublime Text 3</strong> - <a href="http://www.sublimetext.com/3?ref=blog.fuzzknuckle.com">http://www.sublimetext.com/3</a></p>
<p>I also install a bunch of plugins for ST3 starting with package control.</p>
<ul>
<li>Package Control - <a href="https://packagecontrol.io/?ref=blog.fuzzknuckle.com">https://packagecontrol.io/</a></li>
<li>Anaconda <em>(Helpful Python Stuff)</em></li>
<li>PlainTasks <em>(Handy plaintext task list)</em></li>
<li>SublimeLinter <em>(install pyflakes, csslint, jshint etc as well)</em></li>
<li>iSort <em>(Sorts python imports)</em></li>
<li>wakaTime <em>(plugin for time tracking website <a href="https://wakatime.com/?ref=blog.fuzzknuckle.com">https://wakatime.com</a>)</em></li>
</ul>
<h4 id="documentationviewer">Documentation Viewer</h4>
<p><strong>Zeal</strong> - <a href="https://zealdocs.org/?ref=blog.fuzzknuckle.com">https://zealdocs.org/</a></p>
<!--kg-card-end: markdown-->]]></content:encoded></item><item><title><![CDATA[Lightup Disco Dancefloor Cake]]></title><description><![CDATA[<!--kg-card-begin: markdown--><p>My sister was turning 40, and my <em>other</em> sister wanted to throw her a party (they share the same birthday, 2 years apart, so she was sort of throwing <em>herself</em> a party too..) It was decided that her and my wife would make a birthday cake, the brief was <strong>&quot;</strong></p>]]></description><link>https://blog.fuzzknuckle.com/lightup-disco-dancefloor-cake/</link><guid isPermaLink="false">64ad387172191c00015744bc</guid><dc:creator><![CDATA[jeff]]></dc:creator><pubDate>Wed, 25 Jun 2014 12:21:57 GMT</pubDate><media:content url="https://blog.fuzzknuckle.com/content/images/2019/02/IMG_0793.jpg" medium="image"/><content:encoded><![CDATA[<!--kg-card-begin: markdown--><img src="https://blog.fuzzknuckle.com/content/images/2019/02/IMG_0793.jpg" alt="Lightup Disco Dancefloor Cake"><p>My sister was turning 40, and my <em>other</em> sister wanted to throw her a party (they share the same birthday, 2 years apart, so she was sort of throwing <em>herself</em> a party too..) It was decided that her and my wife would make a birthday cake, the brief was <strong>&quot;Disco and Peacocks&quot;</strong>. While they were discussing decorations and disco balls, I said &quot;we could do a light-up style dancefloor on the top tier...&quot; and so began the mission. This was decided at late notice, so I had a deadline of one week to get the dancefloor built so that the cake could be constructed around it.</p>
<iframe width="560" height="315" src="//www.youtube.com/embed/s5jWruebwY8" frameborder="0" allowfullscreen></iframe>
<p>Because of the short notice, I could only use stuff I had on hand, but luckily I have a lot of junk laying around. The dancefloor itself would be 3d printed in white PLA, and the lighting would come from WS2812 LED strips, with an arduino pro-mini making the lights blink.</p>
<p>I measured my WS2812 strips, and designed the dancefloor in <a href="http://freecadweb.org/?ref=blog.fuzzknuckle.com">freeCAD</a>.</p>
<script src="https://embed.github.com/view/3d/jef79m/3d-shits/master/dancefloor.stl"></script>
<p>I probably should have measured the bed of my 3D printer, because this took up every mm of space, any bigger and I would hav had to modify my printer a bit. The print actually failed overnight, but I couldn&apos;t spend the time to print another, so the channels to hold the strips were never completed. After cutting and laying out the LEDs in strips of 10, I soldered them up and hot-glued them into place on the plastic floor.</p>
<p><img src="http://i.imgur.com/F2VnJ77.jpg" alt="Lightup Disco Dancefloor Cake" loading="lazy"></p>
<p>It was about this point that I realized I wanted to have a disco ball rotating above the floor as well. I modded a cheap micro servo for continuous rotation, and printed a mount to hold it under the dancefloor. A hole was drilled throught the floor to allow an acrylic rod to pass through, the rod was connected to the servo gear by a piece of vinyl tube.</p>
<script src="https://embed.github.com/view/3d/jef79m/3d-shits/master/servo_mount.stl"></script>
<p>The servo mount was designed with slots for screws, but when it came time to put it together I was too lazy to find screws and just hotglued it all together.</p>
<p>The code was based on the demo from adafruit&apos;s neoMatrix library, and just cycles through each effect function. The effects themselves operate one frame at a time and return a time when the next frame should trigger, this let me define severaleffects that run at different speeds.</p>
<script src="https://gist.github.com/jef79m/86fb5846fb72f326b6f2.js"></script>
<p>Now that the functionality was there, I added in a couple of details. I didn&apos;t want to have a visible switch on the cake, so I put a small reed switch on one side of the floor and connected it to one of the arduino pins, this let me set the cake up and then pop a magnet on top to switch off all the leds and the motor, minimizing battery draw. Because I would be running this from a LiPo battery, I didn&apos;t want it draining my battery to the point of destruction, so I made a quick voltage divider and connected that to one of the arduinos analog pins, and switched the code to run some predefined &quot;low battery&quot; effects so I knew to disconnect the battery. As it was, the battery lasted all night and I never got to see the low battery warning.</p>
<p>After that, an 8 inch styrofoam cake blank was hollowed out to hold eveything and I passed it over to my wife who actually did the &quot;cake stuff&quot; to finish it off. In the end, my sister was pretty happy with her cake and I&apos;ve decided that cake decorating is Way Too Hard.</p>
<h3 id="partslist">Parts list</h3>
<p>Arduino pro mini <a href="http://www.ebay.com.au/itm/Pro-Mini-Atmega328-5V-16M-Replace-ATmega128-For-Arduino-Compatible-Nano-With-Pin-/351071006351?ref=blog.fuzzknuckle.com">eBay</a><br>
Adjustable power regulator <a href="http://www.ebay.com.au/itm/281281309959?ref=blog.fuzzknuckle.com">eBay</a><br>
WS2812 LED strip <a href="http://www.ebay.com.au/itm/5M-150-300-Leds-WS2812B-WS2811-RGB-LED-Strip-60LED-M-Individual-Addressable-5V-/251450025290?ref=blog.fuzzknuckle.com">eBay</a><br>
Reed Switch <a href="http://www.ebay.com.au/itm/281073232872?ref=blog.fuzzknuckle.com">eBay</a><br>
Turnigy Micro Servo <a href="http://www.hobbyking.com/hobbyking/store/uh_viewItem.asp?idProduct=30272&amp;ref=blog.fuzzknuckle.com">Hobbyking</a><br>
Turnigy 2650mAh 2s Battery <a href="http://www.hobbyking.com/hobbyking/store/uh_viewItem.asp?idProduct=35438&amp;ref=blog.fuzzknuckle.com">Hobbyking</a><br>
Acrylic rod, local hobby store<br>
Mini Disco Ball, local $2 shop<br>
a couple of resistors to make a voltage divider<br>
PLA plastic filament</p>
<!--kg-card-end: markdown-->]]></content:encoded></item><item><title><![CDATA[Musical Wedding Box]]></title><description><![CDATA[<!--kg-card-begin: markdown--><p>Some good friends of mine got married last year and instead of gifts, they collected donations for charity.<br>
The groom asked if I could rig up a treasure box to play music whenever it was opened. This seemed like an interesting task so of course I said yes.<br>
He found</p>]]></description><link>https://blog.fuzzknuckle.com/musical-wedding-box/</link><guid isPermaLink="false">64ad387172191c00015744bb</guid><category><![CDATA[I]]></category><dc:creator><![CDATA[jeff]]></dc:creator><pubDate>Mon, 27 Jan 2014 10:43:22 GMT</pubDate><media:content url="https://blog.fuzzknuckle.com/content/images/2019/02/image-2-1.jpg" medium="image"/><content:encoded><![CDATA[<!--kg-card-begin: markdown--><img src="https://blog.fuzzknuckle.com/content/images/2019/02/image-2-1.jpg" alt="Musical Wedding Box"><p>Some good friends of mine got married last year and instead of gifts, they collected donations for charity.<br>
The groom asked if I could rig up a treasure box to play music whenever it was opened. This seemed like an interesting task so of course I said yes.<br>
He found a really nice timber box and I started hacking. Being such a nice box, I didnt want to make any permanent changes, so i built a false wall in one side.<br>
<img src="https://blog.fuzzknuckle.com/content/images/2019/03/image-2.jpg" alt="Musical Wedding Box" loading="lazy"></p>
<p>I mounted some neodymium magnets into my false wall and added some small screws into the side of the box, now the false wall is removable and is held in place with magnets.<br>
On the back of the wall, I installed a cheap mp3 player module from dx.com, the amplifier and speaker from a promo phone speaker i got from somewhere, and single cell from a 2200mah lipo that had a dodgy cell.<br>
<img src="https://blog.fuzzknuckle.com/content/images/2019/03/image-3.jpg" alt="Musical Wedding Box" loading="lazy"></p>
<p>To trigger the circuit, I pulled the reed switch from a door sensor and mounted it in a hole drilled in the top of the false wall, the switch is held NO by another magnet held in the lid. When opened, the mp3 player powers up, and automatically plays a song from a microsd card.</p>
<iframe width="560" height="315" src="//www.youtube.com/embed/dPPAqKLlBUY" frameborder="0" allowfullscreen></iframe>
<!--kg-card-end: markdown-->]]></content:encoded></item><item><title><![CDATA[ADS400Q Repairs]]></title><description><![CDATA[<!--kg-card-begin: markdown--><p>After an unplanned landing that snapped the camera platform off the front of my <a href="http://realfpv.com/400Q.html?ref=blog.fuzzknuckle.com">ADS400</a>, I needed to find a way to remount my gopro to the frame. Given that a replacement plate would take too long to get here (shipping from Florida to NSW takes roughly 3 years), I</p>]]></description><link>https://blog.fuzzknuckle.com/ads400q-repairs/</link><guid isPermaLink="false">64ad387172191c00015744ba</guid><category><![CDATA[quad]]></category><category><![CDATA[ads400q]]></category><category><![CDATA[3d printing]]></category><dc:creator><![CDATA[jeff]]></dc:creator><pubDate>Mon, 27 Jan 2014 08:55:07 GMT</pubDate><content:encoded><![CDATA[<!--kg-card-begin: markdown--><p>After an unplanned landing that snapped the camera platform off the front of my <a href="http://realfpv.com/400Q.html?ref=blog.fuzzknuckle.com">ADS400</a>, I needed to find a way to remount my gopro to the frame. Given that a replacement plate would take too long to get here (shipping from Florida to NSW takes roughly 3 years), I fired up my shitty 3d printer and sketchup&apos;d my way to a new (dare i say, improved) camera mount.</p>
<p><img src="https://lh5.googleusercontent.com/-96XhYpnvQeM/UuYavtP1ItI/AAAAAAAAA_4/ChLd8wrJ7Mg/s591-no/IMG_0223.JPG" alt="ADS400Q" loading="lazy"> And, It&apos;s pink!</p>
<p>I knocked up the mount in Sketchup, and tapped the holes so I could replace the forward nylon spacers with my mount.</p>
<script src="https://embed.github.com/view/3d/jef79m/3d-shits/master/gopro_mount.stl"></script>
<p>The finished product, before tapping the holes to accept m3 screws.</p>
<p><img src="https://lh5.googleusercontent.com/-a2qJjFJH2_c/UuYbQfiyQlI/AAAAAAAABAQ/5Nslq9DJ7Pw/s591-no/IMG_0225.JPG" alt="finished mount" loading="lazy"></p>
<p>I haven&apos;t flown it yet, I&apos;m waiting on new props, but hopefully it will be strong enough to stand up to my style of umm... flying.</p>
<!--kg-card-end: markdown-->]]></content:encoded></item><item><title><![CDATA[An adventure in Django and Celery and Redis and Messaging]]></title><description><![CDATA[<!--kg-card-begin: markdown--><h1 id="djangocelery">Django + Celery</h1>
<p>Task queuing and asynchronous jobs, + user notification</p>
<p>Within our project we have identified the following issues:</p>
<ul>
<li>Some tasks take a long time to run (generating reports being the most obvious)</li>
<li>Optimisations so far have only been able to reduce the larger reports to single digit minutes (still far</li></ul>]]></description><link>https://blog.fuzzknuckle.com/an-adventure-in-django-and-celery-and-redis-and-messaging/</link><guid isPermaLink="false">64ad387172191c00015744b9</guid><category><![CDATA[django]]></category><category><![CDATA[python]]></category><category><![CDATA[celery]]></category><category><![CDATA[redis]]></category><dc:creator><![CDATA[jeff]]></dc:creator><pubDate>Tue, 14 Jan 2014 02:52:38 GMT</pubDate><content:encoded><![CDATA[<!--kg-card-begin: markdown--><h1 id="djangocelery">Django + Celery</h1>
<p>Task queuing and asynchronous jobs, + user notification</p>
<p>Within our project we have identified the following issues:</p>
<ul>
<li>Some tasks take a long time to run (generating reports being the most obvious)</li>
<li>Optimisations so far have only been able to reduce the larger reports to single digit minutes (still far from responsive as far as the user is concerned)</li>
<li>Due to the nature of the reports, as more services are added, the generation of these reports will take longer.</li>
</ul>
<p>One proposed solution is to move the generation of such reports to an asynchronous task runner to free up the user to do &apos;other things&apos; while the reports are generated.</p>
<h2 id="celery">Celery</h2>
<p><a href="http://www.celeryproject.org/?ref=blog.fuzzknuckle.com">http://www.celeryproject.org/</a></p>
<blockquote>
<p>Celery is an asynchronous task queue/job queue based on distributed message passing.    It is focused on real-time operation, but supports scheduling as well.</p>
</blockquote>
<p>Given the above, Celery seems to fit our needs fairly well. Generation of reports can be queued and users notified when the reports have been generated, this leaves us with a couple of options to choose from:</p>
<ul>
<li>How to notify the user?
<ul>
<li>In browser?</li>
<li>Via Email?</li>
<li>???</li>
</ul>
</li>
<li>How to deliver the file?</li>
<li>Via expiring link for direct download</li>
<li>Send the document as email attachment</li>
<li>Insert the document into the users securedocs</li>
</ul>
<p>Celery also has the option of working with various backends to handle the storage of task queues, the most common of which being <a href="http://www.rabbitmq.com/?ref=blog.fuzzknuckle.com">RabbitMQ</a> and <a href="http://redis.io/?ref=blog.fuzzknuckle.com">redis</a></p>
<blockquote>
<p>Redis is an open source, BSD licensed, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets.</p>
</blockquote>
<p>RabbitMQ doesn&apos;t have a handy copypastable tagline, but it is a higher level messaging system for applications.</p>
<p>I favour redis, as it is a lower level data store and can possibly be utilised for other needs within the project (ie. browser messages, discussed later).</p>
<h2 id="exampleimplementation">Example Implementation</h2>
<p>An example implementation has been committed at</p>
<p>This example uses redis as the backend for celery, and provides a system to inject messages into the Django Messaging Framework.</p>
<p><strong>System Setup</strong></p>
<p>We need to install redis-server to hold our tasks and messages, so:</p>
<pre><code>sudo apt-get install redis-server
</code></pre>
<p>That&apos;s it as far as redis setup goes. Further configuration information can be found at <a href="http://redis.io/topics/config?ref=blog.fuzzknuckle.com">http://redis.io/topics/config</a></p>
<p><strong>Celery Setup</strong></p>
<p>We can install celery and its required libraries for redis using pip</p>
<pre><code>pip install celery[redis]
</code></pre>
<p><strong>Code Walkthough</strong></p>
<p>More complete information can be found <a href="http://docs.celeryproject.org/en/latest/django/first-steps-with-django.html?ref=blog.fuzzknuckle.com">here</a>. But what follows is a simple explanation of my implementation.</p>
<p>Celery requires a <em>celery.py</em> settings file at the project level (same level as <em>settings.py</em>) This file instantiates the celery instance and iterates through installed apps looking for a <em>tasks.py</em> file to load tasks.</p>
<p>We then register the celery instance in the project&apos;s <strong><strong>init.py</strong></strong>, so that the instance is created when django starts, and is usable through all apps.</p>
<p>At the application level, we have <em>blog/tasks.py</em> which holds the tasks to be created. I the example, <em>busywaitTask</em> simply waits for a number of seconds, then completes.</p>
<p>To start the task, I have created a view that does nothing but start the task. <em>blog/views.py:StartTaskView</em> extends the generic detailview and simply adds the task to the celery queue:</p>
<pre><code>busywaitTask.delay(self.request.user.username)
</code></pre>
<p>That&apos;s it.  The task has been added to the queue and will be dealt with by the celery workers.</p>
<p><strong>Messaging</strong></p>
<p>As an aside, I needed an easy way to notify a user when their task has been completed. The Django Messaging Framework provides a neat way of showing messages to users in the browser, but unfortunately provides no hooks for apps outside of the session to use it.</p>
<p>This was failry trivial to work around using django middleware. I created a simple middleware class <em>middleware/tests.py:getMessagefromRedisMiddleware</em> and added it to middleware classes in <em>settings.py</em>. By overriding the <em>process_request</em> method we have access to the session, and can add messages using the framework. There is a function <em>addUserMessage()</em> which serializes a message and message level and pushes it to the redis db with a key of the sessions username. Then on every request, we lookup the key of the username, and pop messages until there are none left, adding them to the regular messaging framework.</p>
<!--kg-card-end: markdown-->]]></content:encoded></item><item><title><![CDATA[What's this shit then?]]></title><description><![CDATA[<!--kg-card-begin: markdown--><p>Having just re-entered the professional workforce after eight years in hospitality, I need somewhere to put my notes and ideas. So a blog seems like the cool thing to do.</p>
<p>I&apos;ve spun up an Ubuntu instance on digitalocean and installed ghost (it seems the trendy thing to do)</p>]]></description><link>https://blog.fuzzknuckle.com/whats-this-shit-then/</link><guid isPermaLink="false">64ad387172191c00015744b8</guid><dc:creator><![CDATA[jeff]]></dc:creator><pubDate>Mon, 13 Jan 2014 08:53:01 GMT</pubDate><content:encoded><![CDATA[<!--kg-card-begin: markdown--><p>Having just re-entered the professional workforce after eight years in hospitality, I need somewhere to put my notes and ideas. So a blog seems like the cool thing to do.</p>
<p>I&apos;ve spun up an Ubuntu instance on digitalocean and installed ghost (it seems the trendy thing to do), so now all thats left is to write things down.</p>
<p>You can expect sporadic posts about Django, web design, remote control things and maybe some electronics crap too, but I guarantee, it will mostly be rubbish.</p>
<!--kg-card-end: markdown-->]]></content:encoded></item><item><title><![CDATA[Towelie Bro Edition]]></title><description><![CDATA[<!--kg-card-begin: markdown--><p>I thought I would try my hand at acro flying, so got hold of a Towelie Bro Edition frame. (<a href="http://www.multirotorsuperstore.com/?ref=blog.fuzzknuckle.com">www.multirotorsuperstore.com</a>). The frame came without instructions, but was easy enough to put together like the photos online. I built it up with stuff I already had, so recycled the</p>]]></description><link>https://blog.fuzzknuckle.com/towelie_bro_edition/</link><guid isPermaLink="false">64ad387172191c00015744bd</guid><dc:creator><![CDATA[jeff]]></dc:creator><pubDate>Fri, 07 Jun 2013 05:52:00 GMT</pubDate><content:encoded><![CDATA[<!--kg-card-begin: markdown--><p>I thought I would try my hand at acro flying, so got hold of a Towelie Bro Edition frame. (<a href="http://www.multirotorsuperstore.com/?ref=blog.fuzzknuckle.com">www.multirotorsuperstore.com</a>). The frame came without instructions, but was easy enough to put together like the photos online. I built it up with stuff I already had, so recycled the HK20A escs and NTM 1350kv motors of my (now disabled) HT-FPV and put them in together with a Naze32 Acro Edition (<a href="http://www.abusemark.com/?ref=blog.fuzzknuckle.com">www.abusemark.com</a>).</p>
<iframe width="560" height="315" src="//www.youtube.com/embed/4pt8YKOzBTk" frameborder="0" allowfullscreen></iframe>
<p>The Naze32 is on default pids, I have just set the looptime to 3500 and increased acc_lpf. The thing is totally locked in. After I first put it together I gave it a go in horizon mode, and it levelled perfectly. I had some issues with a motor stalling, but i think that was due to a dodgy esc connection, and that problem has since been solved.</p>
<p>A few days later, I charged up some more batteries and went for another go, this time abandoning horizon mode and using acro mode only. I can&apos;t believe how easy this combo is to fly. Immediately responsive and rock solid when holding attitude.</p>
<p>Getting the hang of it, can&apos;t wait to learn how to warthox so I can warthox while I warthox.</p>
<!--kg-card-end: markdown-->]]></content:encoded></item><item><title><![CDATA[Crash Compilation]]></title><description><![CDATA[<!--kg-card-begin: markdown--><p>Here is a compilation of some of my unplanned landings.</p>
<iframe width="560" height="315" src="//www.youtube.com/embed/CZBZ2UOkdiA" frameborder="0" allowfullscreen></iframe><!--kg-card-end: markdown-->]]></description><link>https://blog.fuzzknuckle.com/crash_compilation/</link><guid isPermaLink="false">64ad387172191c00015744be</guid><dc:creator><![CDATA[jeff]]></dc:creator><pubDate>Mon, 06 May 2013 03:46:00 GMT</pubDate><content:encoded><![CDATA[<!--kg-card-begin: markdown--><p>Here is a compilation of some of my unplanned landings.</p>
<iframe width="560" height="315" src="//www.youtube.com/embed/CZBZ2UOkdiA" frameborder="0" allowfullscreen></iframe><!--kg-card-end: markdown-->]]></content:encoded></item><item><title><![CDATA[Lighting the way]]></title><description><![CDATA[<!--kg-card-begin: markdown--><p>Tonight I rebuilt the quad and attached some cob LEDs I had. There are 4 5W lights on the front arms of the quad and I can switch them on via the Tx. They are really quite bright and provide tons of light for the gopro to see quite far.</p>]]></description><link>https://blog.fuzzknuckle.com/lighting_the_way/</link><guid isPermaLink="false">64ad387172191c00015744bf</guid><dc:creator><![CDATA[jeff]]></dc:creator><pubDate>Thu, 07 Mar 2013 11:45:00 GMT</pubDate><content:encoded><![CDATA[<!--kg-card-begin: markdown--><p>Tonight I rebuilt the quad and attached some cob LEDs I had. There are 4 5W lights on the front arms of the quad and I can switch them on via the Tx. They are really quite bright and provide tons of light for the gopro to see quite far. I never looked through my fpv cam but I think its even more sensitive, so it should be good. Next step is getting the balls to fly fpv in the dark...</p>
<iframe width="560" height="315" src="//www.youtube.com/embed/Xe5Pf9Qhkbw" frameborder="0" allowfullscreen></iframe><!--kg-card-end: markdown-->]]></content:encoded></item><item><title><![CDATA[Near Miss]]></title><description><![CDATA[<!--kg-card-begin: markdown--><p>Always check that your battery is charged before flying. This was a close call, and I wasn&apos;t even using my own gopro, lucky the reeds caught my copter.</p>
<iframe width="560" height="315" src="//www.youtube.com/embed/_AB8wMHcgTI" frameborder="0" allowfullscreen></iframe><!--kg-card-end: markdown-->]]></description><link>https://blog.fuzzknuckle.com/near_miss/</link><guid isPermaLink="false">64ad387172191c00015744c0</guid><dc:creator><![CDATA[jeff]]></dc:creator><pubDate>Wed, 06 Mar 2013 02:56:00 GMT</pubDate><content:encoded><![CDATA[<!--kg-card-begin: markdown--><p>Always check that your battery is charged before flying. This was a close call, and I wasn&apos;t even using my own gopro, lucky the reeds caught my copter.</p>
<iframe width="560" height="315" src="//www.youtube.com/embed/_AB8wMHcgTI" frameborder="0" allowfullscreen></iframe><!--kg-card-end: markdown-->]]></content:encoded></item><item><title><![CDATA[Learning To Fly]]></title><description><![CDATA[<!--kg-card-begin: markdown--><p>As part of my self diagnosed adult ADHD, I change hobbies like I change underwear. Currently my addiction of choice is remote controlled multirotors. By taking 3 or more motor / prop combos and applying enough power, you can make basically anything fly. Of course, learning about one hobby tends to</p>]]></description><link>https://blog.fuzzknuckle.com/learning_to_fly/</link><guid isPermaLink="false">64ad387172191c00015744c1</guid><dc:creator><![CDATA[jeff]]></dc:creator><pubDate>Wed, 06 Mar 2013 02:27:00 GMT</pubDate><content:encoded><![CDATA[<!--kg-card-begin: markdown--><p>As part of my self diagnosed adult ADHD, I change hobbies like I change underwear. Currently my addiction of choice is remote controlled multirotors. By taking 3 or more motor / prop combos and applying enough power, you can make basically anything fly. Of course, learning about one hobby tends to blend back with existing ones, so combine a flying machine with a camera and the result can only be good fun.</p>
<p>So in learning to fly I have also had to try my hand at video editing, obviously neither are much good right now but I&apos;m sure with a bit of practise I&apos;ll have something I&apos;m proud of.</p>
<iframe width="560" height="315" src="//www.youtube.com/embed/6dnknywI2u4" frameborder="0" allowfullscreen></iframe><!--kg-card-end: markdown-->]]></content:encoded></item></channel></rss>