Graphing the commute time to Uni from different possible wellington flats

2018-03-23 | Python , Html , Web ,

Why?

Basically, during my flat hunt one of the most important criteria for choosing a place was the distance and commute time to uni. Ideally somewhere close enough that I could walk to uni is the best option, but I dont mind taking a bus. Living at home, my commute was a 45 minute train ride plus a 15 minute drive to the train station, and a 20 minute walk from the station.

How?

How do I get a list of addresses for flats that were on the market? Trademe! Trademe offer an API to do this very easily, but from my understanding you need to apply for a license to be able to use the api on the main website. Now, I didn't want to do that just to run a one-time script so I did the next best thing: web scraping. Using Beautiful soup I was able to pull the street address off of each listing on a filtered trademe search.

Next, I had to use the google maps api to give me the travel time to the uni from each flat. Because google maps can only take so many queries at once I had to split each call up into chunks of 5 or 10 addresses. I did this for both transit(Bus/Cablecar/Train) and walking. For us, the most relevant chart was walking as we really want a place that is within walking distance.

Then, using that, I graphed the results with Matplotlib:

The graphs

Weirdly, the first graph seems to show a big chunk being under 25 minutes and then a big chunk after that. I theorise that this could be because of the line between 'inner city' flats and 'outer city flats' - ie outside the CBD area. Alot of student accommodation in Karori is over 25 minutes away - I imagine this is likely due to all the flats that are 25 minutes away are all rented already and people like the location.

update 15th July - Well, alot of the bus routes have changed as of today, but I theorise the results are still mostly relevant.

Where to go from here?

I could try making a graph of price versus travel time and try look for correlations. Also, I could take different readings throughout the year and try see if the available flats get further away as time goes on and students move into the close ones.

Wheres the source code?

When I was installing linux on my surface tab apparently bitlocker was enabled (???) and triggered after I changed the partition size of windows causing the tablet to lock down completely. After multiple calls with microsoft support it turns out they dont store the keys anywhere to get my computer unlocked. In all my infinite wisdom I forgot to check the source code into source control so now ive lost it... hopefully theres a backup somewhere on my network.

My smart home, more server stuff, and plans for IoT

2018-02-20 | IoT , Python , Arduino , Html , Web ,

This post is a work in progress.

What I have now

I have a bunch of random stuff around that I plan on connecting together in a DIY IoT-thingy. While I am still looking for a flat this project is pretty much on hold for installing permanent fixtures but I can still experiment in my test environment and get a basic setup going.

Currently the components I have are:

• 3 raspberry pi's
• millions of ESP8266 Nodemcu kits (these things are awesome)
• two old security cameras
• an Amazon Echo Dot
• a few temperature sensors
• motion sensors
• a bluetooth speaker
• 5 metres of RGB LED striplights
• arduino's everywhere
• 433mhz power outlets!

What am I going to do with all of this, I hear you ask?

Here is a list of all the things that I plan on doing right away:

All of the sensors feed into a central hub that controls everything. This is Home Assistant and its freaking awesome. Its open source and made in python so is easy to configure and extend. My current home page looks like this (it's a bit basic of course):

todo put a photo here 6/3/18

I recently got an amazon echo from Kordia as a leaving gift. I plan on using it to add voice automation to home assistant - everything that can be controlled through the web interface should be voice controlled too. Alexa - the echo's voice - can be extended through the use of skills, which can be developed for free. There are already a bunch on the amazon store.

The Server

I recently got myself a new rackmount server - a Dell R170 poweredge. It has 64 gigs of ram, 16 cores, 5 odd hard drives and all the other redundant power supply goodness. Hopefully I should be getting a new managed switch as well to handle most of the networking in the flat. The server is running proxmox but i'm thinking about switching to VMware to be more like the industry standard.

Some of the things the server does currently and will do in the future:

• Plex - video streaming to every computer in the house (I have about 2tb of movies and TV shows currently)
• Pihole - network wide ad blocking (even on phones)
• Couchpotato & Sonarr - automatic downloading of TV and movies
• Deluge - manual centralised downloads
• Wiki (confluence) - centralised wiki notes for uni, the flat, homelab documentation
• Nextcloud - self-hosted fileserver like dropbox
• VPN - access any of our computers at home from anywhere in the world
• Jenkins - CI build server
• Web server - hosting my website and anyone elses
• Game servers - arma, tf2, etc etc
• Calibre - self hosted ebook server
• Gitlab - git repository (like github).
• Pfsense - router
• Home Assistant - IoT home automation hub

Here's what the flat lab will look like:

What this website is

2017-07-06 | Html , Python , Web ,

I’m writing this 6 months after I finished the website so you’ll have to bear with me. Somehow i’ve survived first trimester at Uni. It’s been a stressful change to my life, to say the least! I’m happy that I have a few weeks off between trimesters to get back to some of my projects.

Anywho, if you’ve seen the sidebar on the main page you will know that this website is my ‘creative outlet’. I started around Christmas 2016 and worked on it after work during the summer holidays. The idea came up when I realised that I wanted my own space to spout random ideas, thoughts and ramblings to the world. Social media almost felt too restricting. With my own website, I could make it my own little world - a place I can modify and personalise to my heart’s content. When I found the student developer pack on github I found that I could effectively keep a webserver going for free for about a year until the free credit ran out.

First Revision, modifying one of the bootstrap base templates:

I have always liked google’s Material Design language and I wanted to mimic it in the design of the website. I went ahead and completely redid the site, and found a nice colour scheme using a palette generator. I ended up with this:

Edit: I just realised the text on the second image. Haha, woops.

After this point I kept the design and added all the other pages. In its current form, this is what the website looks like!

This new site has a bit of javascript to keep things interesting, and css link effects stolen from Codrops. I added bits and pieces here and there to the design and also have added a ‘favourite projects’ box right on the front page.

The front-end is cool and all, but the backend is the favourite part of this website. Using sqlite3 I designed a database to store all the posts and even made a dashboard that only I can access to edit and add new posts.

This is all done with flask. The hardest part of this website was probably designing the blog - it sorts the posts by post date and has pages to show 5 posts at a time. I also included a tag filter to reduce the posts down. I could probably open source the page but I don’t want to open myself up to any security holes. Overall i’m happy with how the website turned out and i’m excited to have a place to show off my projects.

A Python/Pygame Robot Coloured Ball Sorter

2015-09-10 | Robotics , python , arduino , pygame ,

Over term 2-3 of 2015, I worked on a ball sorter that allowed me to quickly send multi-coloured ping-pong balls to different outputs depending on their colour. This took me and my partner roughly 40+ hours of development and I submitted it for a few NCEA standards as part of my school technology and robotics course. The project is mostly finished - I planned on doing a few improvement to the colour recognition algorithms and remaking parts of the structure, and then exams hit, so I couldn't complete it. We submitted this project towards Catalysts Arduino Academy competition, winning us first place and a $50 gift card.

The original plan was to use an analog colour sensor paired with an arduino, but it turned out to be nearly impossible to program (and also incredibly boring). After much consideration and looking into other methods I settled on using a dedicated raspberry pi with a webcam as well as the arduino, communicating through serial over USB.

How it works now:

The Raspberry Pi (2, Model B) is connected to a Playstation Eye webcam. I use the python library PyGame to analyze the colour of the middle 50x50 pixels of the camera and take an average of the colour (as shown in the picture to the right, the average colour is displayed in the top corner). From there, I can retrieve the Hue, Saturation and Value (HSV) of the colour - if the saturation is over a certain amount, we know that there is a ball in front of the camera. The script checks to see if the hue is within the range of values for the each of the four colours. From there it takes a modal average of the colours after 50 ‘readings’ and sends the most common colour to the arduino through USB serial.

The arduino waits for a serial input and then moves the funnel servo to a certain position depending on which colour is received. It then moves the gate servo to an open position and then back to let a single ball through. Whilst doing so, the arduino updates the count of balls that has passed through the system and updates the LCD screen. The arduino also makes use of the arduino-academy provided piezo buzzer for playing a startup sound. It was supposed to also play a sound after each colour was sensed (with varying pitched based off hue) but I ran out of time due to mock exams.

I tried to keep everything modular - the RPi controls the colour sensing, whilst the arduino does everything else. The arduino will work without being attached to the raspberry pi (for instance connecting it to my laptop for debugging), although the python code on the RPi will throw an error if there is no arduino connected. All the functions in all the code is also modular and should work by themselves although I haven’t tested this in depth.

I only have some pictures from an earlier point in development due to forgetting to take photos. In the first picture, I have the TFTLCD screen from the Arduino Academy attached the UNO. The screen is displaying the amount of balls going through the system (I had not yet hooked up the servos). The other two pictures are of an earlier prototype. I have since remade most of the structure using our school laser cutter, and changed from the original Arduino Mega to the Uno.

I have since improved the code so that the machine sorts balls faster.

Zombie Game - a 2D Python Platformer

2017-01-01 | python , pygame , zombies ,

Made for my level 3 computing class, my un-named zombie based game is a two-dimensional platformer, horribly optimised for older computers and laptops and with resources ripped straight from CS:GO and random internet sites. That being said, it’s quite fun, and can be a challenge. Featuring randomly generated levels, increasing difficulty, a high score system and more, managing resources to keep away the horde is fun (for a good 5 minutes).

My teacher initially wanted a calculator. Figuring it was way too easy a challenge and somewhat uninteresting, I decided to make a game instead. I bounced around ideas with my friends for days, going through ideas such as Mike Sivali: Zombie Hunter (In which the deputy rector of our school hunted zombies), Mike Sivali: Zombie Fighter and later Mike Sivali - Zombie, and later decided that it may be slightly unethical to use one of our school teachers as a protagonist for a game.

I’ll skip out the boring bits, and just link the build log that I was required to keep throughout the development of the project: Dev Log

The code is available on github here. Go nuts.