Android

Editor’s Note: This post was compiled and created by Scott Andreas, who can be reached on twitter: @cscotta.

Last month, we set a goal of improving the throughput of our Android messaging stack. This stack has three primary components, whose interactions are mediated by Beanstalk queues:

1) The frontend API to which app publishers push messages.
2) A worker responsible for preparing these messages for delivery to devices.
3) Our “Helium” cluster – the edge nodes to which Android devices maintain a persistent connection for message delivery.

Previously, we’d discussed the design and implementation of our “Helium” server, a Java NIO-based service designed to maintain active connections to up to 500,000 devices and deliver messages to them in real time. In this post, we move one level up in the stack to look at our message prep tier.

This tier is responsible for six primary tasks: Delivering individual messages, delivering broadcast messages for a particular app to every user, delivering messages to every device registered with a specific “tag” for an app, storing pending notifications for devices not online at the time of a push, registering and activating devices as they come online, and recording billing stats reported by our Helium cluster.

An Ambitious Goal

Our initial goal was to be able to deliver a sustained 10,000 messages per second with at most five servers. After setting this goal, we moved to analyze the performance of our (then-)current “Heworker” implemented in Python using Celery. The data access pattern for this task requires large, sequential reads for key/value data, which we read briskly from Cassandra, with supporting data fetched from a MongoDB collection. After data is received, the task prepares two protobufs, encrypts a message, wraps it in an RPC envelope, and passes it on to a Helium server.

We found that on an EC2 c1.medium High Compute instance, our present broadcast message delivery rate was roughly 312 messages per second, with delivery to tags at 280 messages per second. After realizing that the vast majority of MongoDB queries could be eliminated by grouping broadcast message delivery into buckets of 20,000 at a time and fetching the supporting data from Mongo with a single “$in” query, we were able to roughly double the throughput of this task to around 622 messages/second, at which point it became largely CPU-bound. Slightly disheartened, simple math suggested that our goal would require at least 16 servers, along with the work of parallelizing delivery of broadcast messages across them.

Enter the Octobot

A few months ago, I wrote, documented, and released a JVM-based task queue worker called “Octobot.” Octobot is a task queue worker for applications that must process message queues with extremely high throughput, minimal latency, and high availability – especially those sensitive to the overhead implicit in receiving and processing many lightweight tasks, such as message delivery systems. Octobot can listen on any number of queues backed by AMQP, Beanstalk, or Redis, with any number of workers for each task queue, with each operating at a configurable priority. Tasks can be written in nearly any JVM language, with sample code available for Java, Scala, Clojure, and JRuby. Configuration is handled by a short YML file, and it runs anywhere Java does. Finally, deployment’s a snap – two jars, an init script, the YML config, and optionally, a logging config and you’re done.

Based on the promising results of a demo, I worked to build out a quick spike of a couple tasks for us to test. Due to more pressing issues, the project was put on hold for some time, with work resuming a few weeks later on a tighter schedule toward a full implementation. We implemented the Cassandra and Mongo data models, ported each of the tasks from Python to Java, and implemented a full test suite using ScalaTest. The re-implementation and testing of this tier took one developer about three weeks.

A Pleasant Surprise

The results were a pleasant surprise. The new Octobot-based worker is capable of delivering about 16,500 tag messages per second, and roughly 13,300 broadcast messages per second per Octoworker instance (a meager c1.medium).  Our original goal of 10,000 messages/second across five or fewer machines could now be handled by just one instance at 75% capacity.

Profiling the Embarrassingly Parallel

During development, profiling Octobot and the worker under VisualVM helped us uncover many opportunities for optimization and parallelization of message delivery. To deliver a broadcast message to all devices registered with an application, we need to know which devices are registered with an application (stored in Cassandra), and to which Helium server they’re connected to (stored in Mongo). The new worker minimizes database and IO latency by delivering messages in batches of 20,000 – that is, by fetching all data required for delivery of a batch of messages, then parallelizing the CPU-bound delivery tasks across a thread pool.

The breakdown of a broadcast to 10,000 devices looks about like this:
– 61ms round-trip to Cassandra to fetch 10,000 keys from an index.
– 257ms round-trip to Mongo to fetch 10,000 records from a collection.
– 432ms to construct the device messages in parallel and place them in an internal queue for delivery.

The final task here is CPU-bound. With all data required for message generation in memory, it is highly parallelizable. As such, we’ve applied a poor man’s fork/join pattern (think map/reduce, but inside the application itself) by dividing the workload across a number of worker threads. This strategy enables us to take advantage of all processors in the system. While EC2’s c1.medium instances expose two cores, the application will parallelize equally well across four, eight, twelve, or more cores. As elsewhere but especially here, we’re thankful to be programming in an environment that allows for concurrency as well as parallelism within a single process.

Parallel Message Delivery Implementation

Wrapping Up

Beyond its raw speed, the Octobot architecture brings significant enhancements to our infrastructure in terms of monitoring and introspection. With a quick Munin plugin, we’re provided with graphs of average execution time, successes, failures, and retries per task (click to enlarge the stats at right). Failures result in an error notification e-mail detailing the incoming task, the hostname of the instance that executed it, arguments supplied, and a stacktrace if one is available. This makes for quick debugging and updates.

All told, we’re very pleased by how the project turned out.  If you’re in the process of implementing a distributed system that requires high-throughput background processing, a similar strategy might be worth considering.

Further Reading

Octobot
If you’re interested in evaluating Octobot for your needs, check it out at http://octobot.taco.cat/ The 1.0 release is just about a week away, with additional samples on the way for more JVM languages, better documentation, instrumentation, and monitoring. It’s processed well over 5 million tasks for us in the past week and is well-behaved.

Fork/Join
If the fork/join pattern is new to you and your work is parallelizable, you might enjoy Doug Lea’s “A Java Fork/Join Framework” (PDF).

Urban Airship
And last but not least, if you’re interested in implementing Push Notifications and remote messaging for your Android application, grab an account! You get a million messages free per month, delivered instantly and securely, to your user’s devices.

Note: Concurrency, as defined in this article, is the same as it is for The C10k problem: concurrent clients (or sockets).

At Urban Airship we recently published a blog post about scaling beyond 500,000 concurrent socket connections. Hitting these numbers was not a trivial exercise so we’re going to share what we’ve come across during our testing. This guide is specific to Linux and has some information related to Amazon EC2, but it is not EC2-centric. These principles should apply to just about any Linux platform.

For our usage, squeezing out as many possible socket connections per server is valuable. Instead of running 100 servers with 10,000 connections each, we’d rather run 2 servers with 500,000 connections apiece. To do this we made the socket servers pretty much just socket servers. Any communication between the client and server is passed through a queue and processed by a worker. Having less for the socket server to do means less code, cpu-usage, and ram-usage.

To get to these numbers we must consider the Linux kernel itself. A number of configurations needed tweaking. But first, an anecdote.

The Kernel, OOM, LOWMEM, and You

We first tested our code on a local Linux box that had Ubuntu 64-bit with 6GB of RAM, connecting with several Ubuntu VMs per client using bridged network adapters so we could ramp up our connections. We’d fire up the server and run our clients locally to see just how many connections we could hit. We noticed that we could hit 512,000 with our Java server not even breaking a sweat.

The next step was to test on EC2. We first wanted to see what sort of numbers we could get on “Small” instances, which are 1.7GB 32-bit VMs. We also had to fire up a number of other EC2 instances to act as clients.

We watched the numbers go up and up without a hitch until, seemingly randomly, the Java server fell over. It didn’t print any exceptions or die gracefully—it was killed.

We tried the same process again to see if we could replicate the behavior. Killed again.

Grepping through syslog, we found this line:

Out of Memory: Killed process 2178 java

The OOM-killer killed the Java process. Having watched the free RAM closely, this was odd because we had at least 500MB free at the time of the kill.

The next time we ran it we watched the contents of /proc/meminfo. What we noticed was a steady decline of the field “LowFree”, the amount of LOWMEM that is available. LOWMEM is the kernel-addressable RAM space used for kernel data. Data like socket buffers.

As we increased the number of sockets each socket’s buffers increased the amount of LOWMEM used. Once LOWMEM was full the kernel (instead of simply panicking) found the user process responsible for the usage and promptly killed it so it could continue to function.

On a standard EC2 Small, the configuration is such that the LOWMEM is around 717MB and the rest is “given” to the user. The kernel is smart about reallocating LOWMEM for the user, but not the other way around. The assumption is that the kernel will use very little ram, or at least a predictable finite amount, and the user should be allowed to go crazy. What we needed with our socket server was just the opposite. We needed the kernel to use all the ram it needed—our Java server rarely uses above a few hundred MB.

(For an in-depth rundown, take a look at High Memory In The Linux Kernel)

On a 32-bit system the kernel-addressable RAM space is 4GB. Making sure the proper space reserved for the kernel is important. But on 64-bit (x86-64) Linux the kernel-addressable space is 64TB (terabytes). At the current state of computing this is effectively limitless, and as such you will not even see LowMem in /proc/meminfo because it is all LOWMEM.

So we created some EC2 Large instances (each of which is 64-bit with 7.5GB of RAM) and ran our tests again, this time without any surprises. The sockets were added happily and the kernel took all the RAM it needed.

Long story short, you can only scale to so many sockets on a 32-bit platform.

Kernel Options

Several parameters exist to allow for tuning and tweaking of socket-related parameters. In /etc/sysctl.conf there are a few options we’ve modified.

First is fs.file-max, the maximum file descriptor limit. The default is quite low so this should be adjusted. Be careful if you’re not ready to go super high.

Second, we have the socket buffer parameters net.ipv4.tcp_rmem and net.ipv4.tcp_wmem. These are the buffers for reads and writes respectively. Each requires three integer inputs: min, default, and max. These each correspond to the number of bytes that may be buffered for a socket. Set these low with a tolerant max to reduce the amount of ram used for each socket.

The relevant portions of our config look like this:

fs.file-max = 999999
net.ipv4.tcp_rmem = 4096 4096 16777216
net.ipv4.tcp_wmem = 4096 4096 16777216

Meaning that the kernel allows for 999,999 open file descriptors and each socket buffer has a minimum and default 4096-byte buffer, with a sensible max of 16MB.

We also modified /etc/security/limits.conf to allow for 999,999 open file descriptors for all users.

#<domain>      <type>  <item>         <value>
*               -       nofile         999999

You may want to look at the manpage for more information.

Testing

When testing, we were able to get about 64,000 connections per client by increasing the number of ephemeral ports allowed on both the client and the server.

echo "1024 65535" > /proc/sys/net/ipv4/ip_local_port_range

This effectively allows every ephemeral port above 1024 be used instead of the default, which is a much lower (and typically more sane) default.

The 64k Connection Myth

It’s a common misconception that you can only accept 64,000 connections per IP address and the only way around it is to add more IPs. This is absolutely false.

The misconception begins with the premise that there are only so many ephemeral ports per IP. The truth is that the limit is based on the IP pair, or said another way, the client and server IPs together. A single client IP can connect to a server IP 64,000 times and so can another client IP.

Were this myth true it would be a significant and easy-to-exploit DDoS vector.

Scaling for Everyone

When we set out to establish half a million connections on a single server we were diving deep into water that wasn’t well documented. Sure, we know that C10k is relatively trivial, but how about an order of magnitude (and then some) above that?

Fortunately we’ve been able to achieve success without too many serious problems. Hopefully our solutions can help save time for those out there looking to solve the same problems.

Today at the Verizon Developer Conference in Las Vegas, Verizon Wireless announced it has selected Urban Airship as its preferred provider of smart-phone push notifications for mobile apps developed across the Verizon network. Today, two of our company founders will take the stage in front of hundreds of Verizon Wireless developers to explain the benefits of push notifications and how they help organizations add value to their mobile endeavors.

We see this as a huge validation of the new mobile channel, especially the potential of push notifications to deliver ongoing value to mobile apps after they’ve hit the market. Businesses that have integrated our AirMail Push mobile messaging services into their apps have realized tremendous benefits, from increased usage to actual revenue gains.

We’re thrilled to be partnering with Verizon Wireless, who like us, seeks to help developers create and enhance the value of their applications. By expanding the number of tools available, including AirMail Push, to members of the Verizon Developer Community, the carrier is providing cost-efficient options for smart phone developers.

Verizon developers: Welcome! We are looking forward to working with you. We’ll be providing interested parties with tools and resources to guide development, produce higher quality applications and capitalize on Verizon’s powerful mobile device distribution. Here is some info to help you get started.

Push Messaging Overview

Push notifications optimize mobile apps. They strengthen user engagement, open up monetization opportunity and provide valuable insight into the effectiveness of mobile campaigns. Using the Urban Airship mobile services platform also helps developers scale their applications.

Engagement
One of the problems you, like any mobile publisher, face, is how to get your users back into your app once it has become available and downloaded onto their devices. You might see some inertia from your users, who might forget about your app and let it sit unused on their devices. A push notification sends a short message to your users—from within the app—that not only provides information relevant to them but also serves as a reminder for them to go back to the app. Your app users win too; because they opt-in to control which push notifications they want to receive and how they’d like to be notified (via badge update, sound or text alert).

Monetization
The more users use your app, the more they engage with your brand, organization or service. This engagement, of course, opens up revenue-generating opportunities for up-selling additional relevant content to users. It also helps your business develop sponsorship and marketing options. (Of course, the push notification itself is not a medium for direct marketing to users; but savvy publishers can turn that increased user engagement into opportunity.) Mobile messages are also the perfect complement to in-app purchase content delivery – if you have additional content that users might be interested in, what better way to let them know about it than directly from the app?

Information
Push notifications provide unique insight into app user behavior. You can track which messages prompt the most response; what times of day are most effective for sending messages; which devices or time zones or platforms spur the greatest engagement. Understanding your user behavior will help you decide how and when to tailor messaging campaigns to specific users or to users meeting certain patterns. This data helps you glean valuable insight to improve your mobile outreach.

Scale
Urban Airship’s cross-platform infrastructure helps you maximize your reach. From one API, you can send millions of messages across the Android, Blackberry and Apple iOS platforms. At the end of the day, you don’t really care which devices your users carry in their pockets; you just want them to use your app.

Technology
Our solid technology is based on the following principles:

• Provide a simple user and developer experience
• Scale
• Deploy easily as a SaaS solution
• Offer one API to reach any device
• Construct a flexible API upon which developers can build
• Leverage the power of the Amazon EC2 infrastructure
• Build a strong technology foundation using Memcache, Cassandra, Python and Java

How We’ll Work With Verizon Developers

Urban Airship will open its AirMail mobile messaging technology to the Verizon developers network. Any developer considering offering push notifications will be able to work directly with our team of engineers. Urban Airship will also provide code libraries and integration and technical support to guide development and ensure that you successfully integrate our AirMail mobile messaging tools. Our developer resource page has tons of info, including API documentation, to get you started.

Why Urban Airship

We’d like to think it is our warm smiles and generous personalities that wooed the Verizon execs, but we hear it is our demonstrated market leadership in mobile – the nearly 1 billion push messages we’ve delivered to more than 60 million unique devices on three platforms—and of course our solid technology and end-to-end mobile services platform.

Got questions?

Contact us however you’d like. Visit our support site or our IRC chat channel (#urbanairship on irc.freenode.net); we have a contact us form on our site; we host support forums… and we even use email (support@urbanairship.com) and the telephone. There is always more than one way to connect.

Within the past few weeks, we announced the availability of our popular AirMail Push service on the Android platform. Since then, response has been overwhelming. Check it out for yourself. Sign up for an account and give it a try if you like – integration involves dropping two snippets of code into your app and takes about 20 minutes. The docs are right here.

Here, we share with you some of the most common questions about integration, deployment and push in general. Remember, our support team is always here to help. If you want to file a support request, feature request, or suggestion, our support site is for you. We also frequent an IRC channel. It’s #urbanairship on irc.freenode.net. And of course, you can email us at support@urbanairship.com.

Some common questions about AirMail Push for Android from developers

Aren’t push notifications already available on the Android?
As covered in TechCrunch, AirMail Push for Android works with 99.8% of all Android devices (anything running 1.5 or newer). Rather than relying on C2DM in Froyo (which only reaches 4.5% of devices) for push notifications on Android, you can integrate AirMail Push to target nearly every phone on the platform. Plus, you’ll be able to send the same notification to apps on iOS and BlackBerry—from one call to our API.

Some background
When Apple announced it was going to be offering push notification services, it was a boon to app developers. Without Multitasking on iPhone 3.0 OS, there was no way for a developer to communicate directly with the app. Now, with Push baked in to iOS, Apple has provided an open socket for developers to communicate directly with the app – plus, it is the only way to communicate with an app that is not running. Apple didn’t provide a whole lot other than the open socket connection, and that’s how our company was formed, by providing the server- and client- side infrastructure for developers to leverage the service. As of today, we have powered nearly 1 billion messages to the iOS platform.

Blackberry wasn’t far behind. They built a similar type of service, and bundled it with their devices, and we again provide the infrastructure to make it work.

The Android platform lacks this type of built-in service. There is nothing baked into the OS that developers can leverage to send push notifications. Many apps appear to have push notifications, but the apps are constantly polling a server, and when a user has all this polling going on with multiple apps, there’s a big drain on battery life. Another drawback is that the notification is nothing close to real time- it just depends on how often the app wakes up to check for new information.

The AirMail Control Panel (also called the helper app) is the missing piece of the puzzle. When a user has it installed, the device is ready to receive notifications as they occur, and our service sends them to the device in milliseconds. Whereas with iOS we forward the message to Apple and with Blackberry we forward to RIM, with Android, we handle the messaging from end to end.

Why do we need to download a helper app to make push notifications work?
We thoroughly investigated many different service models, and we feel the
stand-alone application approach is best for both the application developer and the end user. Our main motivation, in terms of the interface and design, was to make things logical to the end user. We wanted a way for end users to control the experience of receiving notifications, so they can choose which apps send them push notification, set the method by which the notifications reach them (alert via sound for example) or disable them altogether. This functionality, enabled by the control panel in Android AirMail Push, is similar to how it works in iOS (and as opposed to BlackBerry and C2DM, which offer no such controls). Plus, we’ll be able to easily expand functionality — for example, quiet time will be added in a version coming soon.

Second, due to the sandbox system in Android, sharing a connection across apps would create a security hole—when you have the first app open the connection, every app on the device would be trusting the first app not to do anything untoward.

For more details on the technical reasons why we chose to launch AirMail as a help app on the Android platform, please read this, from our CTO.

Why do I want to add push notifications to my app in the first place?
Push notifications help you communicate with users. Using AirMail Push mobile messaging, you can interact with the users who have installed your apps on their mobile devices, be they Apple iOS, Android or BlackBerry devices. You can contact them in real-time, instantly, with breaking news updates, messages from their friends, sports scores or stock moves – the uses are endless. Plus, whether the phone is carried in a pocket, handbag or briefcase, your message reaches users where they are. This direct-to-pocket experience provides enormous potential for your content.

How does AirMail Push impact the battery life of an Android device?
AirMail Push lowers your app’s battery profile relative to polling and enables instant delivery. You can push a message plus a payload triggering about anything you want. It’s pretty cool. We haven’t noticed any drop in battery usage during our (very long) testing phase. That was one of our primary concerns

How many connections can your servers handle?
Each instance of the layer of our infrastructure that manages the connections can handle several hundred thousand connections.

Can you describe the integration with your Android libraries?
You can see the integration story around the push library here. We provide a drop-in library for you that communicates with AirMail Control panel via intents. The library is up on GitHub.

Does your app handle the notification or does it just broadcast an Intent?
Our app handles notifications by default, but if you don’t want it to handle the alerting and want to do it yourself (this was a requirement by some of our larger customers) you can by specifying only an “extra” value. We’ll pass that data along and your app can parse it.

Are the messages secure?
Yes, all notifications are encrypted.

Can UrbanAirship use the Google push architecture for 2.2 devices?

We can, and may, shift our libraries to do so as C2DM matures and is available on more devices, but for now our goal is to offer a singular solution that works well across all devices.

Editor’s Note: This post was compiled and created by Scott Andreas, who can be reached on twitter: @cscotta.

Building AirMail Push for Android’s Infrastructure

We’ve been working to develop AirMail Push for Android, our push platform, along with a supporting server-side infrastructure that can handle millions of concurrently connected devices. Like Push Notifications on Apple’s iOS platform, AirMail for Android maintains a persistent socket connection to our cloud, which lets us push out messages to devices in real-time. This brings with it all the complexity of handling hundreds of thousands of persistent socket connections server side, managing the connection on the client side as users pass in and out of coverage or switch between networks, and detecting and closing half-open sockets as quickly as possible.

This article describes the challenges we faced building out the server-side push infrastructure, how they led us to re-design and implement it using a hybrid threaded/evented/queue-based architecture based on Java’s NIO libraries, and a retrospective on this decision now that it’s live.

Lots and Lots of Sockets

In order to push a notification to a device, you need to have a persistent socket connection open to it. Our platform needed to be capable of serving millions of simulatenously-connected devices, which means that our server’s system stack and our software need to elegantly accept, read from, and push notifications out to a very large number of open connections per node. But how many? 10k? 50k? 100k? 500k? 1MM? More importantly, how many nodes do we need to handle this? If one node can only serve 10,000 clients, the venture is cost-prohibitive from the start – just one million devices would require a fleet of 100 servers.

We began our venture into uncharted territory with an implementation in Python based on Eventlet. Eventlet’s a fantastic library that provides support for concurrent task execution through lightweight coroutines. Its socket implementation allows developers to accept and manage an arbitrary number of connections with less difficulty and expensive context switches associated with threaded implementations. With it, the first version of our edge server “Helium” was born.

Eventlet helped us build a Helium whose code was incredibly straightforward, clean, and easy to understand. We love Python and are very grateful for the community of developers who are working to build libraries like this. Our initial implementation of Helium enabled us to accept and send application-level keepalives to roughly 37,000 connected clients before hitting the 1.7GB memory limit of a base EC2 instance. While pleased with this number, the amount of instances needed to meet our capacity requirements would have been prohibitively large and expensive. While we are constantly investing in our infrastructure to build a stable, secure, performant platform, we’re better programmers than we are sysadmins and prefer to write software that scales both efficiently and cost-effectively.

Things That Look Like Sea Monsters

Thus began our search to build a prototype of a more efficient socket server. We considered a variety of options, including a C/C++-based implementation wrapping [e]poll, a threaded or evented implementation in Java or Scala, a spike using Node.js, along with a few others. As expected, the purely-thread-based Java option fell over after a few thousand connections, and the C-based implementation proved too low-level to meet our evolving project requirements.

A quick experiment using Java’s NIO library (“new/non-blocking” IO) immediately turned heads, so we spent several hours fanning out that spike to include three versions:

• A raw Java NIO implementation
• An implementation of the same, atop the Netty library.
• And one more using Netty, but written in Scala rather than Java.

Each of these three spikes vastly surpassed our previous results both in terms of connections achieved, and memory efficiency:

Table 1: Connections Handled Before Failure

Table 2: Memory Efficiency per Connection

Work Smarter, Not Harder

Jumping from 35,000 connections per node on an EC2 Small instance to over half a million on a single EC2 Large marked a huge win in terms of efficiency, infrastructure cost, and administration overhead — the number of connections per node jumped by nearly 1500%. (As a premature epilogue, this number held true after implementing all of our application logic, keepalives, and message passing, which is almost unheard-of in an early test).

As an interesting sidenote, the failure modes we saw in both the Java + Netty and Scala + Netty were unusual; CPU usage spiked to 100% in the Java process, with nothing output to the console. It appeared that an exception was being thrown and silently caught in a tight infinite loop.

Based on these numbers, we pressed on with a Java + Pure NIO implementation. Stay tuned – we’ll have a few posts on our NIO implementation, more info on our methodology and metrics, some things we’ve learned on the way, and all sorts of sharp corners we’ve hit our heads on that you can avoid.

More soon!

As mobile developers, one of the most challenging obstacles we face is finding ways to effectively market your applications. Unless your application makes it into the coveted ‘featured’ section of the Android market, you’re on your own.

For this reason Urban Airship now provides developers with a new tool called AirDrop. AirDrop which will aid developers in promoting their applications by making it easy to distribute your applications to your friends, fans, and reviewers.

Here are some ways you might use AirDrop:

• Seed reviewers
• Provide free copies for promotions and or contests
• Share them with people interested in your application
• Send development copies to testers

Once you upload your application to AirDrop, you can create custom redeem codes and links to share and distribute. You’ll be able to see who has redeemed codes and, best of all, we make it extremely easy to get the application onto the user’s device.

If the user clicks on the redeem link from their desktop, AirDrop provides that user with some options on how to navigate to that link from their mobile device. Options for sending the url from their browser to their device include a QR code, Google Talk (aka Jabber), and email. Once the user clicks the redeem code from their mobile device the download starts immediately.

Getting started with AirDrop is easy. There is no sign-up, just login with your Google account, upload your application, and start generating redeem code links for distribution immediately.

Just another tool from Urban Airship to help make your lives easier.

Proper spelling is crucial for the written word, especially when having a party. Probably the word I look up most is Hors D’oeuvres, a standby in the Evite economy. Another social spelling necessity is RSVP (all caps? periods after the letters? And what does it mean, anyway?) Luckily, when I’m texting out my invites, I have the dictionary.com iPhone app as a handy reference to these and other social dilemmas.

No, one simply could not host a proper party without the dictionary.com iPhone app.

There’s a Word for That

The iPhone app’s cool Word of the Day Feature also helps out in the chitchat hour of the party, when engaging in small talk. Say for instance, someone is describing her latest culinary triumph, a spatchcock chicken. You probably wouldn’t the slightest clue what she means unless you chose to receive the optional Word of the Day Push Notification from dictionary.com, in which case you’d know right away that “spatchcock” means “fowl that has been dressed and split open for grilling.” Then you could respond with a witty retort.

I venerate the Word of the Day, which, it so happens, is delivered via Urban Airship’s AirMail Push service. We are thrilled to be working with Dictionary.com to enrich the vocabularies of the millions of app users on the Apple iOS and Android platforms by delivering a fresh cool word every day. The free Dictionary.com app delivers world-class reference content from Dictionary.com and Thesaurus.com including nearly 1,000,000 words and definitions and 90,000 synonyms and antonyms.

Dictionary.com push notifications don’t only help users learn new words. They also help dictionary.com with its mobile business strategy, enabling the company to interact with users in real-time, directly to their pockets. And in a way that is non-meretricious because it is opt-in.

Just like a gifted party host, the app is well mannered.