Kicking off 2016 with good Ted talks. This is good one on organizational design and how to simplify it.
Kicking off 2016 with good Ted talks. This is great one on product design bring biology and technology together to build new products.
If you have not already ready my post on zulily engineering blog, it is a good read on what we are building at zulily. Send me a note if you want to learn more or if you are interested in joining our team.
In July 2014 we started our journey to building a new data platform that would allow us to use big data to drive business decisions. I would like to start with a quote from our 2015 Q2 earnings that was highlighted in various press outlets and share how we built a data platform that allows zulily to make decisions which were near impossible to do before.
“We compared two sets of customers from 2012 that both came in through the same channel, a display ad on a major homepage, but through two different types of ads,” [Darrell] Cavens said. “The first ad featured a globally distributed well-known shoe brand, and the second was a set of uniquely styled but unbranded women’s dresses. When we analyze customers coming in through the two different ad types, the shoe ad had more than twice the rate of customer activations on day one. But 2.5 years later, the spend from customers that came in through the women dresses ad was significantly higher than the shoe ad with the difference increasing over time.” – www.bizjournals.com
Our vision is for every decision, at every level in the organization, to be driven by data. In early 2014 we realized the data platform we had which was combination of SQL server database for data warehousing primarily for structured operational data + Hadoop cluster for unstructured data was too limiting. We started with defining core principles for our new data platform (v3).
I need to read more about these 5. Apache Flink, Apache Samza, Ibis, Apache Twill and Apache Mahout-samsara. Mahout is the one I have read a bit about but others were not on my radar yet.
There are a lot of open source projects out there, and keeping track of them all is next to impossible. Here are five important ones in the Big Data space that you may not know about.
This is one of the best books on management I have read in a while.
There are twelve questions in first chapter that managers should focus on to attract and retain top talent which are really good. These are like “Do I know what is expected of me at work?”, “Do I have materials and equipment to do my job?”, “Do I have opportunity o do what I do best?”, “ Have I received recognition in for good work?” etc…
According to author,
- The role of manager is to act as catalyst by working one on one with each person to convert his talent into performance.
- Also, there is limit to how much you can develop people hence managers should help each person to become more of who he or she already is.
- Focus on outcomes not right steps
- When hiring select for talent, skill and abilities
- Talent cannot be taught, skills and knowledge can be developed
- Each person has his own strength, focus on strengths and work around weaknesses
- Most importantly “FIND THE RIGHT FIT”
Great read, highly recommend it.
This is a great read for folks who are interested in online marketing and analytics. Traditionally most online ads have been sold based on unique views or cost per click(CPC) model. There is a new trend where companies are looking at selling these ads based on time spent.
CPC & view based model has introduced lot of phony sites who only try to get you to come to the site and show you ads and make you switch between pages to show more ads.
Hmm, Interesting question is what are the scenarios where time based model makes sense? Recently I have seen lot of sites add “Sponsored Links or content”. This is a great scenario for time based revenue model. If people read the whole article advertiser pays more money, if they bail immediately you . Another example would be to charge more when someone watches longer video ad on a site.
Analytics focused companies like Omniture and Chartbeat are spearheading the revolution and will be interesting to see which other models are introduced and if any of them will actually threaten the traditional powerhouses like Google. I am sure CPC model is not going anywhere soon but exciting times ahead…
In 2014 digital subscriptions to the Financial Times grew 21 percent, more than twice the rate of overall circulation, and the digital audience accounted for 70 percent of paying readers. Still, the salmon-colored business newspaper faces the same problem as all publications: Print ad rates are far higher than digital ones. But FT Group Chief Executive Officer John Ridding says its secret weapon is audience engagement. “Our audience is not just growing,” he told shareholders on Feb. 27. “People are spending more time with the FT and consuming more content.” To take fuller advantage of those eyeballs, the paper’s website has started to change the way it sells ads.
Author: Leo Babauta
I had high hopes from this book. It has 4 stars on Amazon and I wanted to read a book that would genuinely help me with how to manage with less. This book was huge disappointment for me.
Actually when you have author giving you morning routine ideas like Have coffee or tea, Shower, Take a bath, Eat breakfast among others ideas, you really think if author is running out of ideas on what to write. Anyway, mostly the book is just common sense stuff nothing that you do not know already…
The only good part was I skimmed through this book REALLY FAST!!!
Good paper on how we leverage Hortonworks Data Platform for our big data processing and benefits we get from this. You should also check out our joint video with Google on leveraging HDP + Google Big Query.
zulily is a publicly traded, Seattle-based online retailer. The company launched in 2010 with a mission of bringing its customers (primarily moms) fresh and interesting product every day. The company has over four and a half million active customers and expects to do over $1B in sales for 2014.
Unlike search-based e-commerce sites whose users come looking for specific items, the zulily customer visits the company’s web site and mobile app on a daily basis to browse and discover products. The company focuses on crafting engaging, unique and compelling experiences for its customer.
The zulily experience promises “something special every day,” and creating that experience is no small feat. To do so, the company’s 500 merchandising professionals create over 100 daily “events” that launch more than 9,000 unique products each day.
To turn this raw content into an engaging experience for customers, zulily has invested heavily in personalization. Showing the right sales event and images to the right member at the right time is critical for zulily. A mother with a 6-year old daughter should have a completely different experience from that of an expecting mother.
…take advantage of the latest technologies available in the industry for managing both structured and unstructured data.
Accomplishing this level of personalization required that zulily build systems to understand members coming to its web site and then instantly determine what to show them. To do this, zulily’s systems must capture, integrate and analyze many different inputs from a wide variety of sources.
The company was founded with a data platform built on a relational database, but two years after launch the number of events, SKUs, customers and interactions were growing too rapidly for that system to keep up. To continue delivering relevant, customized content to its rapidly growing customer base, zulily knew that it needed to modernize its platform.
For personalization at scale, zulily built a Hadoop-based system for collecting clickstream data from across the company’s web, mobile and email engagement channels. This system allowed the company to turn clickstream data into engines that produce personalized, targeted and relevant product recommendations.
The zulily platform helped it achieve a new level of precision and maturity in its ability to personalize its customers’ experiences, but one challenge still remained. It still had a silo of structured data in one place (including transactions, customer records and product details), which was separate from its clickstream data in Hadoop.
“We really struggled to integrate and analyze the data across the two different silos,” said Sudhir Hasbe, director of software engineering for data services at zulily. “We often found ourselves making decisions based exclusively on a single type of data, or needing to get developers involved to produce new reports or analyses.”
The need to constantly involve the company’s development team was expensive and time consuming, and it distracted the developers from focusing on their own priorities. Due to the complexity of its siloed data platform, the company found itself limited in its ability to agilely respond to changes in the marketplace or company strategy.
After zulily thoroughly examined its analytical priorities and the challenges posed by its current infrastructure, the company concluded that it needed to move beyond its legacy relational database.
This platform is allowing us to take questions that we couldn’t answer to the point where we can answer them
“We knew we couldn’t build what we needed on traditional relational database technology,” said Hasbe. “We would have to innovate and take advantage of the latest technologies available in the industry for managing both structured and unstructured data.”
As Hasbe and his team further defined the company’s requirements, they formed a vision for what the company now calls the zulily Data Platform (ZDP). They planned to make ZDP a primary, central repository for all of the business’ data. It would:
- Support the company’s efforts to enhance the customer experience through better personalization and targeting,
- Give the company’s business analysts easy access to all of the company’s information,
- Allow the team to make smarter business decisions without needing IT support, and
- Scale to support the company’s growth over the long term.
To meet these goals, Hasbe and his team created a modern data architecture that combined the strengths of both Apache Hadoop and cloud computing to deliver a highly scalable unified data platform for structured and unstructured data.
ZDP is based on:
- Hortonworks Data Platform (HDP). With YARN as its architectural center, HDP provides a data platform for multi-workload data processing across an array of processing methods – from batch through interactive to real-time, supported by key capabilities required of an enterprise data platform – spanning governance, security and operations.
- Google Cloud Platform (GCP), Google’s public cloud infrastructure-as-a-service (IaaS) offering.
- Google BigQuery, a cloud-based tool for super-fast data query across massive data sets.
- Tableau, a visualization and reporting tool suite.
After identifying its path forward, the zulily team was able to move from its vision to an in-production data platform in a mere four months. It will migrate all existing data processing to the new platform by the end of 2014.
“Our new platform enables analytics scenarios that were difficult to achieve with our former technology stack,” said Hasbe. “And we now have the ability to scale both storage and analytics on demand. To finally be ahead of the company’s growth curve is exciting for us.”
For Luke Friang, zulily’s chief information officer, the ZDP platform creates important new opportunities for the company.
“Data is everything to us, yet we really struggled with how to properly consume and harvest a mass of data to provide our customers with a great experience,” said Friang. “Our new platform empowers us to use data all over the business. It drives the content of the email that our customers receive in the morning. It drives how and when we ship customers the products they order. It drives what customer sees in the mobile app versus what customer sees in a browser on their computer. It’s allowing us to make sure that we’re tailoring customer experiences appropriately, throughout the entire lifecycle as a zulily customer.”
From Friang’s perspective, it all comes down to supporting the business’ ability to derive new insights and make quick decisions.
“This platform is allowing us to take questions that we couldn’t answer to the point where we can answer them,” he said. “It is allowing us to accelerate decision making processes from weeks, days and hours to minutes, seconds and milliseconds. From an off-line analytics activity to a real-time decision-making processes embedded within a piece of software. That’s the value.”
“Hortonworks’ depth of knowledge was invaluable to us in this process,” added Friang. “The responsiveness of their team, and their ability to get things done and get issues fixed, were key to our ability to get ZDP off the ground.”