Tagged: code

Hakology – How to grab openweathermap data using python.

I’ve been playing with the openweathermap api thought i’d share a little code, the following pulls the json and constructs a string containing the location, temp, tax temp, min temp and humidity. You will need to sign up for an API key and free user account 1st. The free account at the time of writing this allows 50k calls a day so if you’re writing for a small personal project 50k calls should be fine.

Copy the code below to a file
eg. displayweather.py

chmod the file if needed …
sudo chmod +x displayweather.py

run the file using …
python displayweather.py

import requests
r = requests.get('http://api.openweathermap.org/data/2.5/forecast/city?id=YOURCITYIDGOESHERE&APPID=YOURAPIKEYGOESHERE')
data = r.json()
mystring = ""
mystring = mystring + data['city']['name'] + " "
mystring = mystring + "Temp:" + str(int(data['list'][0]['main']['temp'] - 273.15)) + "C "
mystring = mystring + "Max:" + str(int(data['list'][0]['main']['temp_max'] - 273.15)) + "C "
mystring = mystring + "Min:" + str(int(data['list'][0]['main']['temp_min'] - 273.15)) + "C "
mystring = mystring + "Hum:" + str(data['list'][0]['main']['humidity']) + "% "
print mystring

Click here to download the file.

Hakology Day 82: Digistump … rickroll all the things.

[0657]
Well merry crimbo, I hope you had a great day whatever you spent it doing. So i’ve been really busy the last few weeks but I’m starting to have a little more free time now … and thought I’d take this chance to start writing up a little code and project for the digistump.


What is a digistump?

A digistump is a small USB development board that emulates a HID (Human interface device (Usually a keyboard or mouse but there are other variants)). The digistump allows the user to flash up to 6k of code to the device which when plugged in to a computer after programming will execute the code on the device as if it were a keyboard and/or mouse.

Why would I use one?

Its very handy for automating small tasks such as downloading a file and running an install or just editing settings on the pc that remain consistent across operating systems. eg. You could use the digistump to run a command in the command window or run a specific application with certain options automatically. All you need to do is plug the device in after programming and it will start executing the pre-programmed keypresses.

What operating systems does it support?
The digistump is cross-platform this doesnt mean one script works for all operating systems. This means the device is capable of running and executing code on Win/Linux/Mac but due to difference across the various operating systems scripts would need to be customised for each. The digistump has no way of reading data or accepting any feedback from the PC it just blindly presses keys. Your scripts will rely heavily on intelligent timing. Some commands will execute and finish on modern PCs faster than they would on older hardware, this has to be taken in to consideration when writing code.

Where can I get one?
The digistump is available from Digistump.com you can also find them available on ebay and similar sites. I bought mine from ebay for £1.50 each which is a tiny amount when compared to similar devices.

Installation?
The install procedure is pretty straight forward, download arduino IDE, install drivers and add digistump examples and templates. Which is all detailed on this page here … Getting started with the digistump
The setup is relatively straight forward.

So why am I reading all of this?

Well b/c the digistump is a relatively new product there’s not a lot of reference material on the internet so I started developing a small framework to make it easy to deploy and develop code very quickly. I’ve been busy working on my first little project for the device and right of passage to rick roll any windows 7 users. Although this project is a harmless bit of fun it’s helping me to develop a lot of standardised functions for running applications, opening web pages creating and saving files.

Rickroll notes …
I spent the first few nights tearing my hair out with this device. Here are some of the issues I encountered and how I mitigated them or formed some workarounds.

The first major issue I had was the backslash. The digistump by default outputs US scancodes, as I live in the UK this was an issue. So after much googling and head scratching I figured out that the scancode for the backslash on a UK keyboard was 0x64 yet the digistump was sending 0x31 the US scancode for the backslash.

Whilst I couldn’t figure out where the digistump library resided on the PC I wrote a small function to swap out the 0x31 for 0x64 which seems to have remedied my backslash issues. This is not the correct way to do things.

Eventually I found the library location last night (c:\Users\Username\arduino15\… ). I’m still yet to look through the code and figure out a conversion table for 101(UK en-gb 32) keyboards. Given a little more time I’ll get this fixed and not have to use any functions for string processing.

Another related problem was the saving of files using the %USERPROFILE% environment variable. Full filenames containing this variable were not being parsed properly ie. the environment variable was being read as %USERPROFILE% and not the actual users name. To mitigate this I broke the file string down in to sections and type each part of the save file string in separately. ie. C:\ [ENTER] Users\ [ENTER] %USERPROFILE%\ [ENTER] etc which allowed me to use the %USERPROFILE% variable when saving files.

I’m not going to upload all the code yet as its still messy and I have some functions that need more calling parameters adding so if I released the code now it’ll probably change before the final release and I want everything nice and polished before I release everything.

A great big shout to advancednewbie who’s been working on a special script for the digistump (More on that very soon.) his research and project helped me greatly in trying to figure out the key mappings for most default buttons and some of the UK differences. Given some more time im sure we’ll have this working seamlessly between countries and keyboard layouts.

Even though these issues don’t directly relate to the rickroll project I thought I’d include them just in case anyone else is having similar issues.

Download digispark digistump stumpyroll source code
[/0657]

Hakology Project Notes : Python, RedditAPI, Arduino, RaspberryPi thing.

Over the last three weeks I’ve been playing with various breakout boards, microcontrollers and embedded linux. I’ve been trying to take time to learn how the technology works and how best to combine them to teach and produce something useful, different and entertaining. So this project initially started out when I was playing with the RTLSDR dongle, I thought it might be cool to display the messages as they got demodulated on a separate device that was dedicated to grabbing POCSAG messages, I slapped it all together, then when my son was over the other weekend we decided to modify the code to grab the top 25 articles from a given subreddit.

Eventually when I get time ill put together a video tutorial on setting this all up but in the meantime here are the notes …

You will need:

1 x Raspberry Pi model B.
1 x Wifi dongle or ethernet cable.
1 x Arduino deumilinove.
1 x USB lead (for serial communication with the arduino)
1 x USB lead (to power the RPi)
1 x USB Charger
1 x LED Matrix 8×8 x4 (I used the horizontal style layout)

So how does it work?
Its pretty simple really, the raspberry pi runs a small python script that grabs the top 25 articles from a predefined subreddit using the api (most of reddits api can be accessed just by post fixing .json to a lot of their URLS the data structures are also well documented and available on github.) The script strips out any weird characters as the JSON is returned in utf-8 format. A string is built containing the articles rank, total amount of upvotes and the article title. If the string is over 300 characters it gets trimmed and “…” appended to the end. The script creates a serial connection to the arduino (the device is usually addressable using /dev/ttyUSB0) The string is then sent to the arduino over serial. When the arduino receives a new string it displays it to the LED matrix after having scrolled any current messages.

What’s the python code?

Install and configure raspbian in the normal way. Before running the script make sure you have internet access otherwise the python script will fail when it trys to communicate with reddit. (todo: add better error handling.)
Python code – http://pastebin.com/shXyry5C
Copy the code in to a file called reddit.py to your home directory.
Edit the file so it points to the correct serial device.
nano reddit.py
If you’re unsure about the arduino serial port run
sudo ls /dev
Look for something like …
ttyUSB0
Edit the line in reddit.py where the serial communication is initialised ensure it matches the port for the arduino.
ser = serial.Serial('/dev/ttyUSB0','57600')
Save the file using …
ctrl+x
Make sure the file has execute permissions by issuing the following command …
chmod +x reddit.py
You can test the script by running it from the command line using the following command (*this needs to be run with superuser privs to access the serial port) …
sudo python reddit.py
Once you’re happy with the way the script is running you can make it autoboot with the pi by using the following command …
sudo crontab -e
If you are prompted which editor to use and are unsure use nano (option 2)
At the bottom of the file add the following line …
@reboot /local/bin/python /home/user/reddit.py
Save the file by pressing ctrl+x
Reboot and test your script is autoloading.

To display the messages on the matrix you’ll need to flash your arduino with a little code …

Wiring the arduino …
To connect the arduino to the matrix you will need to ensure you have the correct SPI pins as they differ between arduino boards the ones on the deumilinove are pins A4 and A5, A4 SDA and A5 SCL, you will also need to attach the CS (chip select) pin to pin number 13 on the arduino and also connect VCC to 5v and GND to GND on the arduino.

A4 to SDA
A5 to SCL
13 to CS
5v to VCC
GND to GND

(todo : will post a pic when i get time)

What’s the arduino code?
Arduino code – http://pastebin.com/pvkM2DUK
Copy and paste the arduino code in to your arduino IDE, you’ll also need to grab the md_max29xx library from git hub. (Download the zip file and unzip the files to your arduino IDE library folder usually /Documents/arduino-x.x.xx/libraries) This library is used to drive the LED matrix. Once you’ve installed the library you’ll have to specify how many matrices you are using in the arduino code. (How many LED matrix panels you will be displaying to my board had 4 8×8 LED panels.) Here’s the important bit though depending on what/where/who you bought your led matrix board from you might find you’ll have to edit way the arduino draws to the matrix otherwise you could be in for some very interesting results. This can be done in the md_max29xx lib folder. There are a few variables that define the matrix rotation and draw direction IE. right to left, left to right, top to bottom etc. You will need to play with these variables until you get the right combination for your matrix.

NB. When initially getting this running use the example grapicstest to make sure the leds are drawing as expected then flash the arduino reddit serial code once you have the library configured correctly.

Now all the hard work’s done
Once you’ve installed the library, edited the arduino code, updated the variables for the draw direction and successfully flashed the code, plug the arduino in to the raspberry pi and boot. There should be enough of a delay in the script to ensure the RPI has connected to your network before it makes its initial request.

If you have the arduino setup correctly you should see scrolling text on the matrix. “Waiting for serial data!” after a short while the rpi should connect and start displaying articles from reddit.

Concluding
Still yet to build an enclosure or decided how / where im going to use this yet but i’ve had fun making it. Initially getting the libraries setup and running correctly was by far the most frustrating part of this project writing / editing the code was pretty straight forward. There’s plenty of scope for improvement / modification too, if I get time I was thinking I’d get it to cycle multiple subreddits, but for the time being I’ve spent enough hours on this project.

NB. The original code running on the arduino was limited to 25 characters I’ve modified this to allow for messages up to 300 characters in length (the size of an article title from reddit) Since doing so some of the serial messages appear to glitch intermittently. I’ve been checking the code for errors but haven’t managed to pinpoint the problem yet. As far as I can tell the python code runs fine but there might be a problem when the arduino is receiving the serial messages. Possible solutions maybe change the baud rate and send the data slower or add a checksum. (*As suggested by advancednewbie) Example code an library for implementing a checksum. https://github.com/AdvancedNewbie/serialChecksum

IF YOU HAVE ANY QUESTIONS REGARDING THIS PROJECT PLEASE PUT THEM IN THE COMMENTS ON THE YOUTUBE VBLOG EPISODE AT THE TOP OF THIS POST.

Hakology Day 43 : Blog Entry

[1714]
Time to get busy NO MATTER WHAT. Had a great weekend spent with my son and family, we even did a little coding using the reddit API. Sunday was cool too, I managed to grab some bargains from a few second hand sales, nothing exciting or tech related, just a few books on survival/bushcraft/scouting. Today im going to attempt to record the SDR# video as the last time I forgot to hit record! (*not quite I forgot to hit record but that’s another story/rant), I’m also ready to go war driving so that might be a thing in a few hours. Had some more breakout / dev boards turn up in the post but not the one I’m waiting for, lol, its always the way.
[/1714]

[2318]
Nothing ever goes as planned, I went for a little war drive earlier after having installed kismet, gpsd-clients and gpsd. Recently I reinstalled debian with lxde and I couldn’t figure out, or seem to kill the gpsd process to set my gps as the default device (/dev/ttyUSB0), no matter what I did couldn’t get kismet to detect the dongle in gpsd or even find the configuration file. I decided to run kismet with the –use–nmea option BUT this was a lot less reliable in kismet than what gpsd usually is. Then my netbook went to sleep as I havent figured out the power saving settings in lxde yet. (I’ve been using lxde less than a week) I pulled over to restart my scan but accidentally chose the wrong wireless device when I restarted kismet, so all in all a complete mess. That said I’m working on a little script to load everything at once correctly so this won’t happen again. I really newb’d out tonight, If at first you don’t succeed try harder.
[/2318]

[0033]
sudo kismet -c wlan1 -use-nmea-gps /dev/ttyUSB0 -gps-reconnect true
DO NOT USE THIS FOR GPS IT SUCKS … I had the most sketchy log files using this syntax to call kismet. Might have been the way I had the dongle positioned but I think its the way netxmltokml process the file when its been generated using that switch.
[/0033]

[0515]

[/0515]

Hakology Day 39 : Blog Entry

[2348]
Progress I managed to get my LED matrix working with the md_max27xx library, going to see if I can get multimon-ng+python the send the last received message to the matrix. Also reinstalling my netbook OS as things have got a little messy.
[/2348]

Hakology – Ubercrypt – Python multi-layer AES encryption.

In this video we’ll be looking at a script I wrote a few months ago to facilitate multilayer aes encryption (cryptographic strengthening) using multiple keys and ivs combined into one monster key. Allows users to encrypt share and decrypt data using the command line. Demo is for linux but this may work if you can install the python crypto library on windows.

Full source code …
https://github.com/caffeinemonster/ubercrypt

sudo apt-get install git python python-crypto
git clone …
python crypto.py -g 4096
python crypto.py -e “Hack the planet”
python crypto.py -d ENC-23-32-54-232

enjoy
have fun hacking !