This tutorial will demonstrate how the Arduino Yun can be controlled from anywhere with any internet connected web browser. Controlling the Yun with a web browser connected to the same network, for example connected to the same wifi router as the Yun, is straight forward and well covered in the Arduino Yun bridge tutorial. However, a common problem in home automation and internet of things applications is that it is difficult to reach devices connected behind wifi routers from the outside. There are different approaches to overcome this problem:
- Port forwarding and static ip addresses. This solution requires the user of the connected device to know how to configure a router and have access to router administration which is not always possible. A Yun tutorial with port forwarding is found here.
- Polling is a technique where the connected device at regular intervals checks with an external server if the device should take action. This solution requires no configuration of the router but it creates extra network traffic and response delays.
- A third way is to use WebSockets which is a way of providing real time full-duplex communication over TCP. Spacebrew is a good open source toolkit for connected devices using WebSockets. Autobahn is another infrastructure that can be used.
- Reverse HTTP is the solution that will be used in this tutorial. We will use Yaler which is an open source relay infrastructure that gives access to connected devices with very little configuration.
Let’s get started.
Step 1: Download all the example code from GitHub
Step 2: Sign up for a yaler account at try.yaler.net and note your yaler relay domain.
Add your relay domain on the second line of the Arduino sketch yunYaler.ino:
[cpp] #define DEBUG false //NOTE: the serial monitor must be opened if debug is true #define RELAYDOMAIN "your-relay-domain" //Change this to your relay domain, something like gsiot-zzzz-yyyy [/cpp]
Step 3: Make sure that your computer and the Yun are connected to the same network. Instructions can be found at the Arduino website. Note: The Yun and the computer only has to be on the same network during configuration. As soon as the programs are uploaded to the Yun, they can be on completely different networks, which is the whole point with this tutorial.
Step 4: Start terminal on Mac or for example PuTTY on Windows. Change direcory to the folder where yunYaler.py is located on your computer with something like
[bash gutter="false"]cd /Users/username/some/folder/somewhere/[/bash]
Step 5: Copy the python program yunYaler.py to Yun with
[bash gutter="false"]scp -r ./yunYaler.py root@arduino.local:/usr/lib/python2.7/bridge[/bash]
and enter your arduino password when prompted.
Note: If you have changed the name of your Yun from the default name arduino, you will have to use root@yourYunName.local:/usr...
instead. You can also use the ip-address of the Yun like this: root@http://192.168.1.96:/usr...
Step 6: Upload yunYaler.ino to the Arduino. Make sure you have changed the RELAYDOMAIN definition to your domain first. Also set the DEBUG definition to either true or false. true requires the serial monitor to be opened for the sketch to run. Debug messages will then be printed to the serial monitor.
The arduino sketch opens a pyhon script yunYaler.py that does the actual communication with the Yaler relay. The python script is launched by the runYaler-function in yunYaler.ino:
[cpp] void runYaler() { p.begin("python"); p.addParameter("yunYaler.py"); p.addParameter("try.yaler.net"); p.addParameter(RELAYDOMAIN); p.addParameter("bridge"); //enable bridge mode p.runAsynchronously(); //makes it possible for the arduino to do other things while waiting for yaler request } [/cpp]
runAsynchronously
makes the python script and arduino sketch run independently from each other, and makes it possible for the arduino to do other things while it is waiting for a yaler request. Yun Bridge is used for communication between the arduino sketch and the python script. A very good introduction to Yun Bridge is covered in the blog Adventures with Arduino Yun. If, for example, a led on pin 13 should be turned on, a REST command is sent to Yun from any web browser:
http://try.yaler.net/your-relay-domain/arduino/digital/13/1
The python script parses out the part of the url after /arduino and sends to the arduino sketch through the bridge with value.put('rest',rest)
:
[python] if acceptable and (x[0] == '1') and (x[1] == '0') and (x[2] == '1'): keepGoing, requestUrl = getStringBeforePattern(' HTTP/1.1', s) pos=requestUrl.lower().find('/arduino') if pos!=-1: rest=requestUrl.lower()[pos+8:] if usebridge: value.put('rest',rest) now=time.strftime('%H:%M:%S', time.localtime()) value.put('time',now) putStatus=True [/python]
The arduino sketch checks periodically if the bridge is updated, and reads the rest command if it is updated:
[cpp] Bridge.get("rest", frompython, 32); String rest=String(frompython); Bridge.get("time", frompython, 32); String time=String(frompython); if (lastTime!=time) { lastTime=time; process(rest,time); } [/cpp]
If there is a new rest command, the function process is called that tries to find a valid command and act accordingly:
[cpp] void process(String rest, String time) { // read the command String command = head(rest); rest=tail(rest); // is "digital" command? if (command == "digital") { digitalCommand(rest); } // is "analog" command? else if (command == "analog") { analogCommand(rest); } // is "mode" command? else if (command == "mode") { modeCommand(rest); } else { String answer="{\"command\":\"unsupported\",\"action\":\"unsupported\"}"; Bridge.put("answer",answer); } } [/cpp]
Step 7: Now that both the Arduino sketch and the Python script are uploaded, all is set and it is time to test it. After power up, it might take more than one minute before the Yun has booted and the wifi connection is established. It is important that the wifi connection is established before the Arduino sketch starts the Python process. To ensure this there is a 2 minute delay when Led 13 will blink before it is ready.
Note 1: Led 13 will blink fast (twice a second) for 1 minute, stop blinkning for
some seconds and start blinking slow (once a second) for another minute
Note 2: if DEBUG is true there will be no delay. Instead led 13 will blink very fast
until the serial monitor is opened. As soon as the serial monitor is opened, the led will stop blinking and it is ready. The Yun must be connected to the computer with a usb cable for this to work.
Note 3: If you forgot to change your-relay-domain to your domain in the Arduino sketch, Led 13 will be solid on (no blink)
Step 8: Just enter the following URL-s in any web browser to read or write data. Be sure to change your-relay-domain to your yaler relay domain:
- set D13 to HIGH (turn built in led on) with
http://try.yaler.net/your-relay-domain/arduino/digital/13/1
- set D13 to LOW (turn built in led off) with
http://try.yaler.net/your-relay-domain/arduino/digital/13/0
- read D13 state with
http://try.yaler.net/your-relay-domain/arduino/digital/13
- read analog input A4 with
http://try.yaler.net/your-relay-domain/arduino/analog/4
The response to the above commands will be in json format:
{"command":"digital","pin":13,"value":1,"action":"write"}
{"command":"digital","pin":13,"value":0,"action":"write"}
{"command":"digital","pin":13,"value":0,"action":"read"}
{"command":"analog","pin":4,"value":679,"action":"read"}
The structure for the rest is as follows:
http://try.yaler.net/your-yaler-domain/arduino/command/pin
to read data or
http://try.yaler.net/your-yaler-domain/arduino/command/pin/value
to write data
command
can be either digital
or analog
, pin
can be 0..13
for digital
and 0..5
for analog
. value
can be 0..1
for digital
.
Enjoy turning on and off an LED from remote!
Complete arduino and python code
UPDATE: Please note that some stability issues have been reported with the original version of yunYaler.py. An updated version with added time out is available at GitHub.
yunYaler.ino
[cpp] #define DEBUG false //NOTE: the serial monitor must be opened if debug is true #define RELAYDOMAIN "your-relay-domain" //Change this to your relay domain /* This sketch makes it possible to control the digital outputs of the Yun connected to a network without having to know the Yun IP number or having to do any port forwarding settings on the router the yun is connected to. This is possible by using the yaler service (try.yaler.net) As a bonus you can also read the analog inputs */ #include <Process.h> String compiletime=__TIME__; String compiledate=__DATE__; String file=__FILE__; Process p; // Create a process and call it "p" const int ledPin = 13; // the number of the on board LED pin unsigned long previousMillis = 0; // will store last time bridge data was checked unsigned long now; const long interval = 200; // interval at which to check updated bridge data char frompython[33]; //data read from bridge that was put by python. // temporary storage before it is compied to a string. Allocate // space for a 32 character long rest command String lastTime; //time when data was read from the bridge void setup() { lastTime="99:99:99"; // set the digital pin as output: pinMode(ledPin, OUTPUT); unsigned long pause; if (DEBUG) { pause=0UL; } else { pause=60UL*1000UL; } now=millis(); //wait for the wlan to get established and everything else to start //fast blink while((millis()-now)<pause) { digitalWrite(ledPin,1-digitalRead(ledPin)); delay(250); } frompython[0]='-'; frompython[1]='\0'; // Initialize Bridge Bridge.begin(); now=millis(); //wait a little more for the wlan to get established //slow blink while((millis()-now)<pause) { digitalWrite(ledPin,1-digitalRead(ledPin)); delay(500); } if (DEBUG) { //this requires a serial cable between arduino and computer. Serial.begin(9600); delay(25); // Wait until serial is opened. Blink fast while doing so digitalWrite(ledPin,LOW); while (!Serial) { digitalWrite(ledPin,1-digitalRead(ledPin)); delay(25); } digitalWrite(ledPin,LOW); } printdebug(compiletime); printdebug(compiledate); printdebug(file); runYaler(); //start the yaler pyton process } void loop() { now = millis(); if(now - previousMillis >= interval) { previousMillis = now; //get the rest data from python Bridge.get("rest", frompython, 32); String rest=String(frompython); Bridge.get("time", frompython, 32); String time=String(frompython); if (lastTime!=time) { lastTime=time; process(rest,time); } } } void runYaler() { p.begin("python"); p.addParameter("yunYaler.py"); p.addParameter("try.yaler.net"); p.addParameter(RELAYDOMAIN); p.addParameter("bridge"); //enable bridge mode p.runAsynchronously(); //makes it possible for the arduino to do other things while waiting for yaler request } void process(String rest, String time) { printdebug("----------"); String temp="Processing ["+rest+"] at "+time; printdebug(temp); // read the command String command = head(rest); rest=tail(rest); // is "digital" command? if (command == "digital") { digitalCommand(rest); } // is "analog" command? else if (command == "analog") { analogCommand(rest); } // is "mode" command? else if (command == "mode") { modeCommand(rest); } else { String answer="{\"command\":\"unsupported\",\"action\":\"unsupported\"}"; Bridge.put("answer",answer); } } void digitalCommand(String rest) { int pin, value; //parse pin number from rest string pin=headNumber(rest); rest=tail(rest); //the part of the rest following the number //parse value from rest string value=headNumber(rest); rest=tail(rest); printdebug("digital command pin: "+String(pin)+", value: "+String(value)); if (pin!=-1 && value!=-1) { digitalWrite(pin,value); String answer="{\"command\":\"digital\",\"pin\":"+String(pin)+",\"value\":"+String(value)+",\"action\":\"write\"}"; Bridge.put("answer",answer); } else if (pin!=-1 && value==-1) { value=digitalRead(pin); String answer="{\"command\":\"digital\",\"pin\":"+String(pin)+",\"value\":"+String(value)+",\"action\":\"read\"}"; Bridge.put("answer",answer); } else { String answer="{\"command\":\"digital\",\"action\":\"unsupported\"}"; Bridge.put("answer",answer); } } void analogCommand(String rest) { int pin, value; //parse pin number from rest string pin=headNumber(rest); rest=tail(rest); //the part of the rest following the number if (pin!=-1) { value=analogRead(pin); String answer="{\"command\":\"analog\",\"pin\":"+String(pin)+",\"value\":"+String(value)+",\"action\":\"read\"}"; Bridge.put("answer",answer); } else { String answer="{\"command\":\"analog\",\"action\":\"unsupported\"}"; Bridge.put("answer",answer); } } void modeCommand(String rest) { //not implemented yet. Needed if you want to change digital pins between input and output String answer="{\"command\":\"mode\",\"action\":\"unsupported\"}"; Bridge.put("answer",answer); } int headNumber(String s) { int number; String numberString=head(s); if (numberString.length()>0) { number=numberString.toInt(); } else { number=-1; } return number; } String head(String s) { //returns text after leading slash until next slash return s.substring(1,s.indexOf("/",1)); } String tail(String s) { //returns text after second slash (if first char is a slash) return s.substring(s.indexOf("/",1),s.length()); } void printdebug(String s) { if (DEBUG) { Serial.print(s); Serial.println(char(194)); } } [/cpp]
yunYaler.py
[python] # Copyright (c) 2011, Yaler GmbH, Switzerland # All rights reserved # based on TimeService.py by Yaler GmbH # modified for Arduino Yun by Bo Peterson www.asynkronix.se # note: this version sometimes freezes on unstable networks. # An updated version with added time out is available at # https://github.com/bopeterson/yunYaler import sys import time import socket sys.path.insert(0, '/usr/lib/python2.7/bridge/') from bridgeclient import BridgeClient as bridgeclient value = bridgeclient() def find (pattern, s): x = [0] * len(pattern) i = j = t = 0 while True: k = 0 match = True while (k != len(pattern)) and match: if i + k == j: x[j % len(x)] = s.recv(1) j += 1 t = x[(i + k) % len(x)] match = pattern[k] == t k += 1 i += 1 if match or (t == ''): break return match def getStringBeforePattern (pattern, s): beforePattern='' x = [0] * len(pattern) i = j = t = 0 while True: k = 0 match = True while (k != len(pattern)) and match: if i + k == j: c=s.recv(1) beforePattern += c x[j % len(x)] = c j += 1 t = x[(i + k) % len(x)] match = pattern[k] == t k += 1 i += 1 if match or (t == ''): break if match: beforePattern=beforePattern[:(len(beforePattern)-len(pattern))] return match,beforePattern def location(s): host = '' port = 80 if find('\r\nLocation: http://', s): x = s.recv(1) while (x != '') and (x != ':') and (x != '/'): host += x x = s.recv(1) if x == ':': port = 0 x = s.recv(1) while (x != '') and (x != '/'): port = 10 * port + ord(x) - ord('0') x = s.recv(1) return host, port def accept(host, port, id, usebridge): rest='{"command":"unsupported","action":"unsupported"}' # if /arduino not found in url putStatus=False x = [0] * 3 while True: s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect((host, port)) while True: s.send( 'POST /' + id + ' HTTP/1.1\r\n' 'Upgrade: PTTH/1.0\r\n' 'Connection: Upgrade\r\n' 'Host: ' + host + '\r\n\r\n') j = 0 while j != 12: x[j % 3] = s.recv(1) j += 1 if (x[0] == '3') and (x[1] == '0') and (x[2] == '7'): host, port = location(s) acceptable = find('\r\n\r\n', s) #find the calling url and parse it if acceptable and (x[0] == '1') and (x[1] == '0') and (x[2] == '1'): keepGoing, requestUrl = getStringBeforePattern(' HTTP/1.1', s) pos=requestUrl.lower().find('/arduino') if pos!=-1: rest=requestUrl.lower()[pos+8:] if usebridge: value.put('rest',rest) now=time.strftime('%H:%M:%S', time.localtime()) value.put('time',now) putStatus=True else: print rest if not acceptable or (x[0] != '2') or (x[1] != '0') or (x[2] != '4'): break if not acceptable or (x[0] != '1') or (x[1] != '0') or (x[2] != '1'): s.close() s = None if not acceptable or (x[0] != '3') or (x[1] != '0') or (x[2] != '7'): break return s, rest, putStatus usebridge=False if len(sys.argv)>3: if sys.argv[3]=='bridge': usebridge=True while True: s,rest,putStatus = accept(sys.argv[1], 80, sys.argv[2],usebridge) time.sleep(0.3) # this should be about 50% longer than interval in arduino sketch if putStatus: reply=value.get('answer') else: reply=rest # url must contain /arduino s.send( 'HTTP/1.1 200 OK\r\n' 'Connection: close\r\n' 'Content-Length: '+str(len(reply))+'\r\n\r\n' + reply) time.sleep(0.001) s.close() [/python]
[facebook_like_button]
Pingback: Internet of things with Arduino Yun and Yaler |...
Pingback: Arduino Blog » Blog Archive » Internet of things with Arduino Yun and Yaler
Pingback: Internet of things with Arduino Yún and Yaler | Faweiz Blog
Is it possible to port this to Raspberry Pi?
I think it already is. Check out http://yaler.net and go to get started pages. There you will find raspberry instructions.
Pingback: Electronic Hobbies | Internet of things with Arduino Yún and Yaler
When i run yunYaler manually, I get this error:
root@Arduino:/usr/lib/python2.7/bridge# python yunYaler.py gsiot-xxxx-yyyy nobridge
Traceback (most recent call last):
File ”yunYaler.py”, line 132, in
s,rest,putStatus = accept(sys.argv[1], 80, sys.argv[2],usebridge)
File ”yunYaler.py”, line 83, in accept
s.connect((host, port))
File ”/usr/lib/python2.7/socket.py”, line 224, in meth
return getattr(self._sock,name)(*args)
socket.gaierror: [Errno -2] Name or service not known
Any idea what is the problem?
Sorry for a late reply but your comment got stuck in the spam filter 🙁
I don’t know why you get the error, but I will see if can replicate it. Does it work when you run yaler through the Arduino sketch?
you have to pass python yunYaler.py try.yaler.net gsiot-xxxx-yyyy nobridge
you had forgotten the ”try.yaler.net” part
Great project!!
I contacted also Tomas Amberg who wrote the Yaler interface and ansered med that Yaler+ Yun is not officially supported …
I think you won!
I’m standing by some develops (I’m not a programmer) like a web interface to act commands through JSon strings :). Good work!
This is a great tutorial – many thanks.
I had a problem where it would work from a web browser, but not a PHP script. In the end realised the problem was at the Yun end. I had to make sure the ”rest” String wasn’t followed by a header and use curl_init() rather than file_get_contents().
Dave
—
Here’s the code:
[php]
String rest = trimAfterWhiteSpace(frompython);
String trimAfterWhiteSpace(String s){\
String trimmed;
int space=s.indexOf(’ ’);
int tab=s.indexOf(’\t’);
if(min(space,tab)==-1){
if(max(space,tab)==-1){
trimmed=String(s);
}
else{
trimmed=s.substring(0,max(space,tab));
}
}
else{
trimmed=s.substring(0,min(space,tab));
}
return(trimmed);
}
[/php]
Great, thanks for the additional Arduino code that makes it work with PHP.
I got the exercise works as expected. The problem I’ve got is it only run no longer 1 hour (approx.), have to cycle power after that. Any idea?
I have experienced the same problem myself. I am working on adding a time out to the python script and it seems to work, but I am not sure. Stay tuned for an update. You can check TimeServiceTimeOut.py at https://github.com/bopeterson/yunYaler to see the idea about adding a timeout.
Great work – thanks a ton!
I think temboo would be very interested in uploading your code (for temboo) on their website and I believe it would be a GREAT addition to the tutorials they already have.
THE ONLY way that makes sense to use a YUN or any Internet Of Things device is with REST – the others usually do some kind of polling which in turn produces a lot of traffic and uses a lot of power as u mentioned above.
Anyways besides loving to see this for temboo I’m very impressed!
Thanks again!
The timeout feature is a great idea. I think only then it becomes a really useful tool – after all, who would want to automate the garage door to only wait and be openable for 1 hour and then be locked out. The Arduino should know the connection is lost and reconnect.
Has there been any progress on the https://github.com/bopeterson/yunYaler at all?
Thanks a ton though – still a great tutorial!
Yes, there has been progress. TimeServiceTimeOut.py has been running for 2 weeks without interruption. yunYaler.py has just been updated with time out at github. Please note that it is not fully tested yet.
The version with time out has been running for many days without freezing. Check github for a version that hopefully is stable. Please let me know if you still experience any problems.
Just had another idea if you are interested:
I think using what u have created over GSM would be great as well!
Cheers
Hej Bo! (I’m a fellow Swede)
Thanks for sharing the code for your project.
I just ordered a Yun (I have a CC3000 that’s not supported by Yaler yet) to get one of my projects of the ground.
During your research did you find that Yaler was the only solution to reach Arduinos behind a NAT?
//Arvid
Hej Arvid
Check the first four bullet points of the post. I list four different methods to reach an Arduino: port forwarding, polling, websockets and the reverse http method that yaler is based on. So the answer is: No, yaler is one of many solutions.
Aside from the ones you listed at the top that is 🙂
I am sure there are more ways. https://temboo.com is a service that works that is not listed at the top.
Hi – I have been following the discussion regarding TimeServiceTimeOut.py and I have installed the latest version on my Yun. However, I’m only get about 6 hours before it stops working. If I restart the Yun (power down) then it works again.
I tried creating a timer on the Arduino that stops and restarts the Yaler process every hour, but that doesn’t help. I’m not sure what’s going wrong or how best to debug it. Any advice gratefully received!
Thanks,
David
Have you upgraded the yun firmware? Have a look at http://arduino.cc/en/Tutorial/YunSysupgrade
I upgraded, and with upgrade and time outs, the yun has been running for weeks now.
Thanks so much – yes, upgrading the firmware did the trick – my Internet of Thing is properly on the Internet! Thanks
Dave
Hi!
Very nice tutorial.Actually I need to connect with Yun my webserver and arduino relay so that anyone from anywhere can swith ON/OFF from my interface web (with my domain and Ip).Is this possible?I can’t use yaler for my project!!!
Many thanks and congratulations!
Beka
If you can’t use yaler, I recommend you try some of the other alternatives mentioned in the four bullet points in the beginning of my tutorial. Try point 1, port forwarding. More info here: http://www.hell-desk.com/remote-heat-control-24-accessing-yun-internet-noip/
how to copy the yunYaler.py to arduino through windows cmd ?
I have no experience of this myself but check http://arduino.cc/en/Tutorial/LinuxCLI
cool
I really appreciate your article
it is really helpful
but isn’t the yaler cost money !!
around $10 a year for one relay !!! what do they mean by one relay ??
is it like one arduino with one relay only ?
or one arduino with more than one relay since you can control all the inputs and outputs
isn’t their a way to do it freely
instead of dealing with yaler ?
thanks
Yaler is free to use. A Yaler relay is not a physical relay that can turn things on and off, but a virtual ralaying service. You can connect many physical relays (electromagnetic switches) to your Arduino and control all of them with one Yaler relay. Also, check the bullet points at the beginning of the post for alternative free ways of doing it without Yaler.
thank you for a quick reply
I really learned a lot from you
now I can do things with the arduino Yun
thanks