Pachube.blog

We have been meeting consistently each month at Meetups in London, NYC, and Amsterdam with like minded enthusiasts to riff together on "Internet of Things topics".  If you are in or around any of these cities, you should come and join us!  This month we heard from all kinds of different projects, some of which you can see in the videos below the break, including:

• Britta Riley from Windowfarms
• Liz Barry from PublicLab
• Vincent Schipper and Alexander Zeh from NetworkLab 
• Amran Anjum and Arnd Marijnissen from the Amsterdam Hackspace
• Stefano Ingarro who came all the way from Turin to talk about EveryAware
• Gonzalo Garcia-Perate from Hintsights
• Haiyan Zhang from Ideo talking about geigermaps
• Ben Redford talking about Olly the smelly robot
• Ben Pirt from Pachube

A new group is starting this month in Barcelona!  Check it out.


Internet of Things London Meetup 1/20/12 from Ed Borden on Vimeo.


Internet of Things Amsterdam Meetup 1/21/12 from Ed Borden on Vimeo.


Internet of Things NYC Meetup 1/12/12 from Ed Borden on Vimeo.

Among the huge amount of awesome stuff going on in London around the Internet of Things, I've had the privilege of putting together some events that bring together a community of people who are really getting their hands dirty.  The vibe and rapport we've got going really is fantastic.  If you are in London, I invite you to come out and join us, but if not, you can get a glimpse of the talks below.


Internet of Things London Meetup 1/20/12 from Ed Borden on Vimeo.

Last month I blogged about a community project called AirQualityEgg aimed at giving citizens a way to participate in the conversation about air quality.  You can read about it here.  You can also join the google group, read/contribute to the wiki, or add your email to this list if you want to know where to go to buy one when we start taking orders.
Credit: Albert Chao
One thing that's been strongly validated in the month since has been an intense interest from communities and individuals all over the world in their local air quality.  People from five continents have reached out to offer their support in various ways, over 150 people have indicated they are interested in purchasing an Egg, and 70 people have showed up to workshops in two different cities .  There are 50 people participating in the google group discussions.  This has obviously struck a nerve.  People want to know.



AirQualityEgg got picked up by Javaun Moradi, a technologist at NPR (American news/media syndicator), who blogged about its implications for journalism. He says:
News organizations are uniquely positioned to serve as ethical overseers...or facilitators of open public dialog [for initiatives like AirQualityEgg]. For lack of a better term, I’ll call this ‘citizen engagement journalism’: applying the newsroom’s tools and values to advance the cause of journalism by means other than reporting. It’s a responsibility that is every bit as noble as reporting and can achieve the journalism goals of informing the public, investigating corruption, speaking for the voiceless, and seeking truth. AJ Fisher, a technologist at a data consultancy in Melbourne, also picked up on AirQualityEgg as part of a concept he's defining as the "Sensor Commons", or "a future state whereby we have data available to us, in real time, from a multitude of sensors that are relatively similar in design and method of data acquisition, [and whose] data is freely available."  This is a fantastic way to describe what is in essence a cultural revolution fomented by the reality of a billion connected people communicating directly with each other and with sensors embedded in the world around them.  AirQualityEgg is right in the middle of this, and I have no doubt that as word of the project spreads, more groups will continue talking about how this type of activism will change their life/work/town/etc.


The biggest criticism of the project has always been around the sensor components themselves.  The project aims to use low-cost, off-the-shelf, easy-to-use components to give us information that is currently only supplied by $50K scientific equipment.  Last month in Amsterdam, over 30 attendees traveled in from 5 countries to attend a one day workshop which aimed to move a step forward on this sensor aspect of the project. There was a fantastic mix between hardware engineers, software developers, and "philosophers" (ie. non-technical participants who wanted to contribute in whatever ways they could).  We were able to break into working groups to tackle different tasks and sensor components, and by the end of the day (after only 4-5 hours of solid work time), we had live data we could look at from a good number of sensors.  We took these outside to test with the exhaust from a Vespa and got a feel for how the output from these sensors would look.  Patrick Beeker, one of the attendees, blogged about how it went here. Groups in Amsterdam and NYC continue to work on this in real time.



Now that we're getting further away from the holidays, momentum is starting to pick back up.  There have been some impromtu gatherings in NYC with some of the attendees of the last workshop and as well as others who have become interested.  Most notably, Albert Chao and Eulani Labay have been charging forward on the Industrial Design of the product.  They've been thinking hard about various aspects of how the system will work and have also been able to run off some prototypes on a 3D printer. While some have questioned the interest in this seemingly gimmicky "egg" concept so early in the life of the project, one of our primary goals here is user engagement for the purposes of building a community.  That requires close attention to details that are quite different than the requirements around evaluating sensor components.

Credit: Albert Chao However, at these gatherings, it's also become clear that the application aspect of the product hasn't yet been given enough consideration. As we've started to tackle the questions of how this sensor network will proliferate, and then, how the network will persist, we keep coming back to what people will actually DO with it.  Moreover, designing the various aspects of the Egg in parallel creates a challenge as each element impacts the rest.  So, as we move toward the zen state where considerations for sensors, product design and the application are in perfect harmony, things are inevitably changing.  Some of the biggest changes:

- The Egg was originally conceived as the sensor enclosure itself. This was perceived as a way to facilitate people building a relationship with this project/movement/concept.  The problem is that it's supposed to be outside gathering sensor data.  So, sensors have been removed to a cheap, nondescript box and the Egg has been re-positioned inside and given a button and an LED.  This gives a way for the user to actually create and receive feedback from the community.  What that actually is has not yet been defined.

- Inside/Outside: The types of pollutants that are inside are very different from outside. It's been enough of a challenge to tackle the outside sensors, and so the "inside air quality" portion of the project is getting nixed for now.  A future extension of the project could address this.

Moving forward, the 30 day goals are the following:

• Create and install prototypes of the sensor unit in NYC and Amsterdam.  We want to see data coming in.
• Define the physical product and how the system works together.
• Define the application and user experience.
• Launch a Kickstarter campaign. This will allow for the gauging of interest and crowdsourcing of the funding for the project in one motion. 

You can help and be a part of this!

• All the functional groups are updating the wiki and the google group asap.  You should join these to keep up to date in real time.  Important discussion will be happening there.
• We will physically meet in NYC to make some decisions about the user experience/application on 1/31 and then again on 2/6 to shoot the Kickstarter video.  These are all open events as part of the IoT NYC Meetup.
• Spread the word.  Share/tweet this blog post!
• Buy an Air Quality Egg! When we start taking orders, I will let you know where to go to buy one if you go and add your email to this list.
  

Thank you to all of the participants so far, including:

Internet of Things Amsterdam Meetup
Casper Koomen
Axel Roest <-- props
Amran Anjum
Bas Mijling
Dorien Zandbergen
Peter Manolescue
Xi SiZhe
Hans Blaauw
Cesar Garcia
Sara Alvarellos
Hans Blaauw
Jeffrey De Sadelaer
Chris Davis
Wout Tankik
Kristof De Buysere
Adrian McEwen
Patrick Beeker
Joseph Timkovsky
Hans Crijns
Ivo van den Maagdenberg
Olivier Brechon
Martin Havranek

Internet of Things NYC Meetup
Eulani Labay
Albert Chao
Volkan Unsal
Joe Saavedra
Leif Percifield

Some new features recently went live!  Unfortunately, a lot of these normally slip under the radar, but as a resolution for the New Year, we're going to be better at talking these up.  After all, our awesome development team is busy making magic... and people gotta know.


- Frozen Feed Triggers

This is one of the most-requested features we've had in the past year. Frozen triggers are sent when there is a 15 minute period of inactivity on a datastream. Live triggers work the same, but are fired when a datastream becomes live (i.e. is updated after being frozen).

One of the best uses for the new frozen and live triggers is make a datastream send a tweet when it becomes frozen. To do that when logged in, first visit the feed page (e.g. https://pachube.com/feeds/504). Next pick the datastream you'd like to be notified for. In the 'Triggers' section, select 'Twitter' from the dropdown menu next to the 'Add a Trigger' link. Then click 'Add a 'Trigger'. In the configuration for the new trigger you may have to log in to the Twitter account you'd like to use. Once that is done you can set the trigger to "Send a tweet when value goes frozen" and customise the text of the tweet. When you're done customizing the tweet text click "Save", and your trigger is all set up. If you want to test your new trigger, click the Debug button and your account will send a tweet simulating the datastream becoming frozen.

- New Docs

The API Docs have been completely reformatted so they are much easier to navigate, view examples, and find stuff.  Even better, the beta feature docs aren't hidden anymore!  Check them out.

- Delete ranges of datapoints

Previously, you could only pass single datapoints to the API for deletion.  You can now specify start and end times and a duration in the event of bad or corrupt data being uploaded to Pachube.  This is documented here.

We talk a lot at Pachube about empowering communities.  The best communities develop important conversations, demand action, curate leadership, and ultimately produce some kind of output that matters.  We've been thinking hard about how our global data platform intersects with global communities and how we can help facilitate this new kind of collaboration around real-time data. This has recently manifested itself in two very simple ways:


- Things tweeting (see our Twitter triggers tutorial video)

Engine bay lights at Crystal Park Fire Station in Manitou Springs, Colorado
Freezing conditions in Dublin, Ireland
Power service availability for coconut oil mills in Perambra, India
In looking through Pachube trigger activity, it's interesting to note that there are quite a few people with Twitter triggers set up to tweet data that is very specifically useful to only themselves.  For example, there might be no context, units, or description whatsoever.  While this is a fine use case, it's interesting to see how in the examples above, people were able to go one step further and translate their triggers into information that has wider implications.  So, instead of "temperature outside my house," @DubFreezeAlert is making a statement about freezing conditions.  Or, instead of "voltage levels from the transformer on my road," @sunish is providing power failure alerts.  He could go even further and make some statement about what areas "Phase 2" affects, etc.

Here's the best part:  Since you are able to place triggers not only on your own datastreams but on other peoples' as well (if they choose to make their data public), if you are interested in a certain datapoint or event related to that data, then you can create the trigger yourself.  So, if you wanted to go and improve @sunish's power alerts in some way, you can simply go and do that.

- Comment threads on feed pages

Questioning the meaning or context of each others' data Learning from each others' experiences
Pachube isn't just about data.  For us, data simply sets the stage or provides the subject matter.  It's the people that add their experiences, goals, and passions into the mix that create richness around the mundane. Without this, the Internet of Things is like a sketched painting without color.  It's incomplete. The most important work -- making sense of it all -- requires participation from people.  We're starting to see that become a reality here.

Pachube Pioneer: Derek Foster
Resides: Sheffield, UK
Occupation: PhD Researcher
Web: http://www.derekfoster.net/
Pachube Feeds: http://pachu.be/24356
Data: Electricity

At the University of Lincoln 60 miles outside of London, there are a range of buildings that vary in age from hundreds of years old to brand new. Derek Foster, a researcher at Lincoln, is working on a project to show that just because some of these buildings weren't designed for electrical systems doesn’t necessarily mean that they must be less efficient than their newer counterparts. Why? It turns out the energy usage habits of the people inside the buildings make a huge difference! He posits: “With the understanding that automated systems can only go so far in reducing consumption, we must adopt the view that people are not just noise when looking at a buildings energy usage, they have a huge impact.”



As an example, this graph shows energy usage in December 2010 compared to the same period the previous year.  Since much of the UK suffered a severe cold snap in December 2010, many people were unable to commute in, resulting in low staffing levels. Derek's theory: "The absence of ‘people activity’ significantly flattened the graph, highlighting the potential of behavior change. Of course it is not possible to completely eliminate this activity, as employees use a range of necessary facilities to carry out their working responsibilities. Rather, we can target a reduction."

After establishing that people (not just automated systems) and how they use energy is important, Derek then moved on to experiment with how to provide them with feedback on their day to day consumption levels. The first hurdle Derek had to overcome was extracting energy usage data out of proprietary storage systems in some dark data dungeon on-site at the University. "It took considerable effort to contact these vendors for technical information, understand the implementation, and then provide a ‘bolt-on’ solution to export this closed data into the Pachube platform,” he said.

The next hurdle was developing ways to easily and usefully communicate this information to occupants of the buildings.

“Although we understand the need for our carbon managers to access energy data with proprietary software for analytical purposes, the full potential and usefulness of energy data can only be realised … we can abstract away from complex energy metrics so employees can understand simpler metaphors for a kilowatt hour, the context of 50 litres of water or the Co2 impact of a 100 cubic meters of gas.

Our work uses social media channels to maximise outreach and impact in raising awareness of energy consumption in an organisational setting. [We have] a Facebook application that displays comparative live energy feedback to groups of friends. See our Wattsup and Power Ballads papers.

We are also implementing energy awareness services through Twitter where you can send a simple tweet to one of our energy handles and receive a response back with more information on the requested building’s energy.  By sending the name of any one of our campus buildings to the twitter handle @lincolnEnergy you will receive a response with a shortened link to that particular buildings 24hr energy graph as generated by Pachube. I also plan on using the bot to post random updates on our energy data. This is just the start and hope to be as innovative as possible with it, time allowing.

It's really all about raising awareness of energy by maximizing social media impact and making the energy data available through as many channels as possible. We hope that our work with open energy data and subsequent publishing on Pachube is seen as the way forward for other higher education organizations in the UK and would be glad to have them on board with us!”

One of the pieces of functionality we've been working away on is a way to make it easy for users to automatically send tweets when triggers are fired. Because this is based on a plugin system, it makes it much easier to build more plugins, so this is the first of hopefully many different options for being able to send triggers via different methods. Let us know if you have any special requests.
If you're an avid Twitter user then you might also find it interesting to create a feed to automatically track your Twitter stats via Pachube. With this plugin you could even automatically tweet when they change! Check out the app here. Check out the screencast for a demo below the break:

Where's the data? Look outside your window -- have you ever wondered what the quality of the air is out there? I mean RIGHT. OUT. THERE. 12 inches from your face. If so, you are out of luck. The air quality data collected by the government is likely sampled from far, far away and then applied to you on a regional level, almost completely useless from the standpoint of trying to understand or change the local dynamics of pollution that affect you. Not good. If you're interested in joining a community of people who are going to change that, read on.



Credit: Albert Chao In the world of December 2011 that we live in, data drives activism. The dialog online has reached a deafening roar and everyone has a cause, so it takes hard evidence to turn heads. Without real air quality data, people can be easily brushed aside, or worse, ignored. But nothing screams, "Take action!" like a link to a datastream updating in real-time showing how people are being affected at this very moment. This is the next form of self-expression, a la YouTube, and it's already happening.

So, why no air quality data?  It's not a technology problem. Open source hardware, data infrastructure, networks -- we have this stuff and can use it to build a cheap networked sensor. How to fund, deploy, and distribute the sensors, and then how to utilize the data we receive -- we can figure this stuff out with a bit of hard work.

The barrier is the naysayers -- people who don't understand that the engine and scale of the internet community has put what we need to solve this problem within our grasp. They are the people who ask why we'd bother, since we don't understand the science. It's true, we do not have access to the technology that builds $50,000 air quality sensor systems, but we do have the ability to put 50,000 $100 sensors systems into a city, a collective voice which won't be able to be ignored. We'll also have a much better view of trends and of what's happening in real-time -- which will most likely tell us things that the official datasets aren't looking for.  Further, if we're a bit savvy about it, we'll be able to create a platform that other people can refine and improve over time. We might not hit the bullseye on the first run through (which may not be necessary anyway if we build an extensible platform), but we'll start an unstoppable movement that will re-shuffle the way issues get discussed.

With this agenda -- to create a way for citizens to participate in the conversation about air quality -- we have partnered with our Pachube communities in NYC and Amsterdam. On November 18, we held a workshop at De Waag, a center for digital media in Amsterdam in the center of the old city, which brought together people from all walks -- hardware, software, scientists, concerned citizens, etc. Led by Casper Koomen, a UX and concept developer, we were joined by Sara Cordoba of design agency Booreiland. Bas Mijling from the Royal Dutch Meteorological Institute provided perspective on local issues and resources. Other speakers discussed projects that helped define our collective understanding of the challenges involved.


Following the event in Amsterdam, another group gathered at eyebeam in NYC on December 2nd, led by Mark Shepard, a well-known artist, architect, and researcher in the Internet of Things space.  Dirk Swart from Wicked Device provided invaluable technical input.  Incorporating the foundation built in Amsterdam, this group tackled the issues around the initial target users, user experience, and methods of deployment/funding.  A consensus on a solution framework and design language which will define the project moving forward was met.  Coming out of that session, a group of committed individuals in both cities are owning slices of this product concept, with the following decisions dictating how development moves forward:

- Ease of use, approachability. The basic unit comes in two parts: a “base station” RF platform that would plug directly into the ethernet port of a wi-fi router, and an enclosed sensor node that communicates wirelessly within 40-50 yards. The wireless node would be protected in a distinct “egg” casing: simple to produce with the potential to create a sense of empathy or stewardship.  The project's name "Air Quality Egg", is taken from this concept.

- Requirement to be able to compare inside/outside measurements. (Should I open the window? Should I get off the bus here? How does the air in my home compare with others?)  Additionally, personal space + public space measuring produces a valuable network effect while delivering immediate personal value.  This is key for both distribution and engagement/participation.

- Platform focus: Applications will be limited and basic at the outset as the design of the network has the enablement of third-party applications at its core.  Furthermore, the hardware will be designed with openness, extensibility in mind.  This group's sensor systems will be the start, but the network will be open to adding hardware (eggs) designed by others onto the core base station.  Have a better way to get at air quality data than the sensor/hardware choices made here? Make your own egg and send it out to the community!

- Crowdfunded: Kickstarter is a natural path for Air Quality Egg.  The project can be funded and simultaneously reach the people who want to be part of this movement. This is a project built, funded, and run by a distributed group of citizens. eyebeam will likely be a key partner at this step.

This is happening.  The Air Quality Egg group intends to prototype, test, and start marketing the product within 60 days.  At Pachube, we're committed to extending resources to keep this alive and on a forward path.

You can help and be a part of this!  

• Come to a workshop in Amsterdam or NYC. We are hacking the hardware in Amsterdam next Saturday, Dec 17. The followup in NYC will be announced shortly thereafter.
• Contribute to various parts of the project, collaboration will be collected on the wiki.  If you have some expertise, particularly in low-cost air quality sensors, we want your brain!
• Spread the word!  Share/tweet this blog post.
• Join the discussion on the open Google Group.
• Buy an Air Quality Egg!  When we start taking orders, I will let you know where to go to buy one if you go and add your email to this list.

Thank you to all of the participants of the workshops so far, including:Internet of Things Amsterdam MeetupOswaldo Heinen
Amran Anjum
Axel Roest
Martin Jacobsson
Rene Pare
Wout Tankink
Kristof De Buysere
Xi SiZhe
Vincent Schipper
Joao Rocha
Roland van Straten
Frans Snik
Stephane Logger
Ben Lawson
Martin Havranek
Casper Koomen
Sara Cordoba
Bas Mijling
Internet of Things NYC Meetup + eyebeam
Art Mikhlin
Tom Lowenhaupt
Adam Griff
Christo de Klerk
Joe Plotkin
Eulani Labay
Volkan!
Janice Kim
Josh Levine
Cindy Konits
Daniel Toke Hansen
Mia Stigsnaes
Eken Ijeoma
Francisco Hui
Albert Chao
Mark Shepard
Dirk Swart
Victoria Marshall

Last month, we made Pachube free for all our users, removing a significant barrier to entry for people and devices to get onto our platform. As another step in that direction, we are today releasing OnBoard, our first turn-key solution aimed directly at device makers of all sizes. With OnBoard, we're making it quick, cheap and easy to offer connected services with internet-enabled hardware. OnBoard includes:1. Device bulk-registration: Manufacturers can provision devices onto Pachube and specifydata profiles.2. End-user account integration: Device owners can now claim their devices and their data directly on Pachube.3. Dashboards: Once devices are claimed, device owners view dashboards hosted within their accounts.Pachube's OnBoard service is a pathway onto the open, people-centric Internet of Things for devices and data – that have in the past have been tied to specific services and software. By breaking open the traditional data-silo/walled-garden approach to building connected services, data can be re-used across many different applications. This increases the value of the data to both its owner and the device manufacturer, while also creating a way for third-parties to develop compatible products on a common framework.

Current Cost Opens Its Data!
We're proud to also announce that Current Cost, a leader in home energy monitoring products, is the first commercial partner to use OnBoard to deliver an online home energy dashboard to their customers. Anyone who has ever purchased a Current Cost monitor with a Current Cost Netsmart (Bridge) device can now access their energy data within their Pachube account by entering the serial number printed on their device here: https://pachube.com/devices/currentcost. You can then:- utilize the new Current Cost dashboard from within your Pachube account- export the data for external analysis- trigger external web services or use Pachube's SMS alert or twitter trigger functions- use applications at http://apps.pachube.com/- use 3rd party applications via the Pachube API- create new applications on top of Current Cost data... you now have keys to the kingdom – full access to and control of your data to do what you need with it!The dashboards are still relatively simple in functionality but expect additions in the near future.If you are interested in building applications that work with Current Cost data, please contact us at biz@pachube.com.
What's Next for OnBoard?Our goal is to make OnBoard entirely automated. Today, if you are a device maker or manufacturer you will need to contact us to get things set up. Once we automate and complete the documentation, we'll release OnBoard officially.With OnBoard we aim to make it as easy as possible to create massively scalable and secure web services that use Pachube, whether you're on-boarding two devices or two million. With more devices on Pachube integrating real-time data with third-party applications, our community will be empowered to build the next generation of device-driven Internet applications.If you're a device maker or manufacturer, and you want to use the OnBoard service while it's in beta, we'd love to hear from you at biz@pachube.com.Find the Pachube OnBoard press release here.

Here's a quick demo of a new feature we're pretty pleased with. The new debug page shows you how many requests you've made to the API in the last 10 minutes as well as a stream of live API requests as you make them.

There's a lot of interesting stuff going on behind the scenes involving message queues, Node.js and websockets but hopefully the end result is easy to use and useful for debugging.

Let us know how it works for you and if there's anything else you'd like to see.  Video below the break.

We’re making the Pachube service free for all users.As of today there will only be one type of account. Every user will have unlimited datastreams, datapoint uploads and history as well as the option to create private feeds. We are setting a limit on the API request rate at 100 requests/minute. Current PRO users will see their rate limit go up from 40/minute to 100/minute. Current PREMIUM users will keep their current rate limit of 250 requests/minute. Any user can request an increased rate limit by contacting support@pachube.com. We will be upgrading everyone’s account in the next few hours. Anyone who has paid for a yearly subscription in the last 60 days will get an automatic refund via PayPal. We made this decision simply because that’s how far back PayPal allows us to go. If you disagree with this decision, please contact us. We’re making this change to make Pachube more accessible to new and existing users, and we want to make sure no one feels treated unfairly.
So, why go free? No, we haven’t decided to become a non-profit. No, we’re not just super-nice. What we are is ambitious. And having worked with and talked with many people in the Pachube community, we know that they’re ambitious too. The “Internet of Things” is a big idea (though secretly many of us don’t actually like that term), and we want our community to lead the effort to define it. As devices continue to find their way onto the Internet, we want them to be able to take advantage of everything the Web has to offer. We want Pachube users to control their own data, build applications that we would never envision, and share with others as they see fit. This idea, as obviously correct as it sounds to us, is not inevitable. There are significant business and technical barriers to this vision. By making the Pachube service free, we’re removing a small barrier today, and we’re committed to removing more barriers in the coming months. The Pachube service is not as easy to use as we want it to be. It still doesn’t do everything we want it to do. This is, of course, okay. That’s why our engineers come to work every day, sit at too-small IKEA desks and code their asses off. Pachube will continue to get simpler, better and more capable. Our intention is that many of these new capabilities will be free. Some of them won’t. We hope that you’ll want to pay for the stuff we decide to charge for.In the meantime, continue to let us know what you need from us, how we can help you build a device-friendly Internet that lives up to your aspirations, adheres to our own lofty ideals, and helps you build the next generation of Internet applications.