Hortonworks has achieved quite a bit of success with online dating. Personally, I haven’t just yet, but hey it warms my heart to think about all those that we’ve helped bring together.
Valentine’s Day is upon us and so I wanted to launch this Cupid’s arrow with a missive about how Hortonworks Data Platform (HDP) is the ultimate hunky, Big Data love machine.
And why not? Big Data is transforming every other aspect of our lives. What about love?
For centuries we remained loyal to St. Valentine and Cupid, but consider how quickly dating has changed! My family used to warn me against meeting Internet strangers. Now it seems I can’t start a family without one.
According to the Pew Research Center, only 15% of American adults have tried online dating, yet usage between 18-24 year olds has tripled since 2013. Clearly we’re beginning to lose trust in Cupid’s arrows as we turn to the Big Data of online dating to find the most suitable matches.
But before we swipe left on St. Valentine, let’s examine these changes in the larger historical context.
A Romanticized History of Valentine’s Day
The romantic notion of Valentine’s Day is widely believed to have sprung up in 14th century. If you’re a history buff like me, you know that’s the same century as Black Death.
The legend tells us that a shot by Cupid’s arrow causes uncontrollable desire. In the 14th century it caused uncontrollable plague, which is why quality control is so important. Centuries later, innovative companies such as Merck use Big Data and HDP to rigorously identify the golden batch to optimize pharmaceutical yields. But I digress.
Were Cupid’s Love Potion #9-tipped arrows responsible for over 50 million deaths in the 14th century? Some say it was rats.
But all facts aside, St. Valentine’s radical notion of celebrating love and devotion just one day a year changed the scene for us singles hundreds of years later. Voilà! By the 15th century the plague was over, demand for flowers resulted in the Wars of the Roses, and by the 16th century a dozen such roses might be marked up three gold pence and a healthy swine on Valentine’s Day.
Looking for Love in All the Right Places, Thank You HDP
The late David Bowie said he was never gonna fall for Modern Love, but Ziggy never met Hadoop. When it comes to picking the person you’ll spend the rest of your life with, an arrow might miss the mark.
Modern data-ing is more pragmatic. It’s still about finding someone of shared values, interests, and ambitions. But now it’s also about using an in-memory data processing engine, such as Apache Spark, to rapidly and iteratively find your soul mate across a social graph of the massive dating pool. Why count on one arrow when you can launch thousands per second?
That’s right, ladies, you heard it first from me. HDFS stores more data than thousands of distributed matchmakers could ever remember. YARN manages those resources to find you a relationship as solid as your HDP cluster: fault tolerant, with a healthy amount of encryption. So whether you prefer to find your life partner at a real-time pace, through interactive query (sometimes quite tedious) or through deep insight into years of dating history, HDP gives you and your emotions multi-tenant comfort.
Security? Check. Governance? Check. An attractive user interface? Yep.
HDP brings it all together so that you can rapidly comb through 200 petabytes of data to find your one-in-six billion companion. That sounds like the odds of winning the lottery, but there’s no luck here. HDP will tell you when, where and with whom to buy your winning ticket. Then the rest is up to you.
We Should Do This Again Sometime
St. Valentine and Cupid were fun for a time, but that time is coming to an end. Until someone actually invents Funky Cold Medina, Big Data can improve our love lives in a big way.