SmoothSpan Blog

For Executives, Entrepreneurs, and other Digerati who need to know about SaaS and Web 2.0.

A/B Testing is a Great Idea for SaaS Startups

Posted by Bob Warfield on May 19, 2016

Tomasz Tunguz (Redpoint VC) and Lloyd Tabb have got it wrong–way wrong.  Tunguz recently published an article based on a conversation he’d had with Tabb that suggests early and mid-stage software companies can’t benefit from A/B Testing because they don’t see enough web traffic to make the results statistically significant.  They suggest that instead, they should make decisions based on qualitative data:

… interviewing users about the whys underpinning their points of view on price, reviewing the video of people exploring the product, and opinions about design. It’s the qualitative data, the acumen of an brilliant designer, the insight of a skilled product manager, the empathy of a master marketer.

Ouch!  Back to anecdotal evidence and marketing decision making by the most important person in the room.  Back to the bad  old days, in other words.  There’s nothing wrong with doing those things they suggest, but before you bet your company on the results, you must A/B test them.  These are just inputs to decide what to test, in other words.


Back to anecdotal evidence and the bad old ways of marketing…

Before we throw the AB Testing baby out with the bathwater, let’s take a closer look at what’s possible.  The Chief Witness for Tunguz and Tabb is Optimizely’s Sample Size Calculator:


Optimizely’s Sample Size Calculator…

It’s a great tool that I use all the time, BTW.  They’ve selected the default view, which suggests that if you have a baseline conversion rate of 3%, and you want to see a minimum 20% detectable effect with 90% confidence, you will need 12,000 visitors to the page.

There are two key questions to explore before we can agree or disagree with the proposition in an informed manner:

  1. Are these the right inputs for Sample Size Calculator?
  2. Given the right inputs, is the sample size too large for most startups to attain?

For the first question, I submit that the defaults are actually not very relevant at all. Requiring 90% confidence or be willing to accept anecdotal evidence is pretty silly.

Heck, I run my own bootstrapped startup, it’s entirely my capital that’s at risk (I’ve accepted no outside investment), and I would be thrilled to ring up 70% confidence interval tests all day long.

As it turns out, Optimizely will only let us go to 80% confidence, but Google’s A/B Testing will tell us it’s evaluation of the confidence regardless of level.  I will add that the statistical confidence is also not the only factor we should consider.  It’s important to make sure you really have a representative sample.  For example, test results may vary by day of the week, so I never accept a test that’s run for less than a week, even if the confidence is 90% or more.  In fact, I typically prefer 2 weeks as a minimum.

Cutting the Optimizely confidence down to 80% gets us down to a sample size of 11,000. Let’s next consider the baseline conversion rate.  3% is not an especially good benchmark for a product landing page. surveyed SaaS companies and came back with a visitor-to-trial conversion rate that averaged 8.4%.

If we plug in an 8% conversion at 80% confidence, the sample size plummets to 3,300 visitors before we can measure a 20% detectable effect.  We’ve cut it almost 4x, but we’re not quite done.  What about that 20%?  Is it not worth conducting A/B tests unless they result in 20% differences?

Here I’ll turn to my own experience AB Testing for my own company, CNCCookbook. In the last 8 months I’ve conducted 55 A/B Tests.  The average change between the baseline and the variant I measured was 30%.  Are you surprised?  I was VERY surprised at how much impact even seemingly little things could have.  FWIW, 44% of my tests yielded a positive improvement, 29% showed the idea failed, and 49% of the tests failed to reach statistical significance.  I have no idea how that compares to the scores for other marketers, but I am very happy with the results.

If we plug that 30% number in, we get to a sample size of 1,300 visitors.  Applying my rule that I usually test for 2 weeks, we need to come up with less than 100 visits a day to the web page we’re testing.

Is that bar too high for startups to clear?  It shouldn’t be if the marketers are doing their job right.  I’m a one-man bootstrapped company and my CNCCookbook site sees about 15,000 views a day to the site.  I get about 250 a day to the home page and about 450 a day to my product home page.  As I write this, Google Real Time Analytics cheerfully informs me there are about 50 people running around on my site.

Clearly I can do very statistically significant A/B Testing and it has benefited me quite a lot. I get over 6000 visits in 2 weeks so I can measure as little as a 15% change in that time, and even less if I am willing to let the tests run longer.  Incidentally, don’t overlook the value of a test that ISN’T significant.  That test is telling you at the very least that even if it is bad, it is no worse than the statistically measurable results.  So, if we can test to a 20% detectable effect, adopting the wrong variant will do no more harm than 20%.  Sometimes when we need to move ahead boldly, knowing we can do no more harm than that is good enough.

Granted, I’ve had this company for a few years, but if I can get this far by myself, a VC-funded startup should be able to do at least as well and much faster.  They have to in order to have much hope for a Unicorn-valuation.  Tabb’s company, Looker, the one that presumably prompted the discussion, looks like it should have a little less than half the organic search traffic I get based on SEMRush results.  Clearly, Looker should be able to benefit tremendously from A/B Testing if it chooses to.

So, VC Board Members–expect quantifiable results from your portfolio companies and don’t take sample size whining for an answer.  Entrepreneurs, saddle up and ride this A/B Testing horse–it’s a powerful tool that can really move the needle.

My best advice for startups right at the beginning, BTW, is start building your audience BEFORE you build your product.  I call it achieving Content-Audience fit, I’ve been writing about it for years, and it is absolutely the very first thing a founding team should do when they get together.  Achieving it provides a number of powerful validations for your team, but more importantly, it validates there is a reachable audience, and in reaching it, you gain a powerful tool for shaping your journey to Product-Market Fit.  Not to mention, you set yourself up to achieve enough traffic to do meaningful A/B Testing just that much sooner.

Stealth Mode is harmful in this respect–it delays your access to Content-Audience fit for no meaningful benefit.  So what if the world knows what broad market you’re working in or even what broad problems you write about?  You don’t have to tell them anything about your product or how it helps solve those problems.

No more excuses–get on with  your marketing people, and do some rigorous AB Testing of it!

Posted in saas | Leave a Comment »

What Really Caused Our Manufacturing Jobs to Move to China?

Posted by Bob Warfield on April 28, 2016

Every now and again, a good rant can be cleansing.  I mean that in the best possible way.

Today, for example, I had a good rant after reading that Fred Wilson, a Wealthy and Influential Venture Capitalist in New York, was “Bothered” by the “Losing Jobs to China Discussion.”

Fred’s not long enough on facts to do much more than be troubled and hand wave away the discussion.  In his mind, there won’t be any manufacturing jobs because automation is destroying them so quickly anyway.


It’s really the Robots that took all the jobs…

Fred doesn’t think it’s worth bothering with the Manufacturing Sector because soon there won’t be any jobs left after automation anyway.  Far better to make everyone an IT guy or some such.  We’re in a transition we should double down on be happy with.  Something like what he says here:

The US and a number of other countries around the world are building new information based economies. That is the long term winning strategy.

So while we can critique our leaders (business and political) for giving up on the manufacturing sector a bit too early, I think the US has largely played this game correctly and will be much better off than the parts of the world that have taken the low cost manufacturing jobs from us.

The thing is, most all of this is a lot of Lies, Half Truths, Myths, and general Bollocks that got started by people who would benefit from offshoring manufacturing and is maintained as a cherished belief as so many myths are just because it’s been a self-fulfilling prophecy. In other words, if we destroyed our manufacturing economy it must be because our manufacturing economy was doomed and not worth saving in the first place.

Take the Robot argument.  It’s uber-popular in VC circles because people like Andrew McAfee have made careers out of pushing this thesis.  Yet, if we actually look at the numbers (which I do in detail in the article below), it’s very hard to make the case that Robots have taken more than maybe 20% of the jobs away.  That’s a far cry from eliminating an entire market segment or deciding they’ll never be able to produce enough jobs to be worth considering.

The reality is the whole thing was manipulated by a variety of parties, is based on a large number of non-truths, and is relatively easy to reverse.  Moreover, it would be extremely valuable to reverse it.

For all the detail, check out my longer post with facts and figures over at CNCCookbook, my own Manufacturing-related company.

It may be that article hit a nerve, because Fred’s site deleted a comment wherein I referred to it as “spam,” LOL.

Posted in saas | Leave a Comment »

Bad Advice on Free Trials from Marketers Who Should Know Better

Posted by Bob Warfield on February 18, 2016


I just read a post on the KissMetrics Blog by Cody Lister entitled, “More Trial Users is Not the Answer For Your Startup’s Growth.”  It’s not a terrible post.  In fact, some aspects are pretty decent.  Lister basically wants startups to focus on better engagement and the onboarding experience and less on just running as many people as possible through the trial process.  He wants you to be sure you’ve got product market fit before you try to scale out with as many trials as possible.  I have no problem with the latter, BTW, but it is unrelated to the areas I do have a problem with.

Unfortunately, perhaps due to the act of trying to make the point as persuasively as possible, he strays into at least one area where I think his advice is dead wrong.  It’s way too Black and White, and whenever someone gives me a Black and White answer, I instinctively look for the exceptions to the rule.  Let’s put aside that in fact more free trials will help you to grow, it simply may not be the optimal thing for you to be focused on right now (or it may be, it all depends).

Instead, let’s drill down on the area that really got me thinking it was bad advice from a marketer who should know better.  Lister states:

Eliminate Or Reduce Free Trials

What if one day, your team just decided to shut down your free trial accounts that were past 14 days since their sign up date? Would you suddenly go out of business?

No, you’d save money from server costs and force people to make a decision.

It’s only when your free trials run out that you know whether the end user found your product worth paying for.

You need to figure out how to improve the engagement of your existing trial users to convert them to paid users.

ConvertKit and Edgar, which today generate millions of ARR, never offered free trials.

I often come across startups that give away free trials for 30 to 60 days. I just don’t get it.

I could not disagree more with his advice to eliminate or limit free trials to 14-days.  He states it as an absolute to the point that, “he just doesn’t get” why anyone would be stupid enough to offer a 30 or 60 day trial.

He gives only two odd exceptions to his rule:

  • A B2B SaaS offering costing more than $200 a month.  No explanation whatsoever why the arbitrary figure of $200 was chosen.
  • An offering where personal data had to be entered and value received increases proportionally to the amount of data entered.  He argues this creates switching costs, which is worthwhile, but actually misses the point.  What he misses is not only does it create switching costs, but the more data in services like DropBox, the more likely the user is to experience the “Aha” moment that closes the sale.  Switching costs come later, after the user is satisfied and someone else wants to woo them away.

Let’s dig into it with a couple of real world examples that I think will help explain the real reasons why you need to think about your Free Trial in terms of the user experience and not in terms of arbitrary advice from marketers.

Ironically, one of the reasons I tried but did not adopt KissMetrics (the very blog where this is posted) was because I could not tell within 14 days whether it would deliver value. In fact, KissMetrics is a wonderful illustration of the problem with this one-size-fits-all advice.

It’s biggest benefit is a better understanding of your sales funnel. So ask yourself, “How far does a user travel in the funnel in 14 days?” Further, how much of the 14 days is needed to get things set up and to accumulate enough people travelling through the funnel to make things even interesting?

You can now see where I’m going.  It might very easily take more than 14 days to get to that “Aha” moment where I see the value in a product like KissMetrics and I’m ready to pay up for it.  In fact, for my company, it really was longer than 14 days.  This was exacerbated by various aspects of the KissMetrics user experience.  It took the service time to accumulate enough data points to show me any funnel reports.  It took me time to understand the service well enough to get my funnels set up properly.  And it took time given my web site’s traffic to accumulate enough data points to see any kind of picture clearly.  BTW, it’s no small web site, I get 2 million uniques a year.  Pretty good for a small business.

I believe 30 days would’ve worked nicely for my case, but alas, I only had a 14-day trial to work with.  So I moved on.

Let’s try another recent personal example: Drip, the marketing automation app.  I wrote about my experience with them recently.  They had a 21-day trial.  During that time I was trying to:

  • Learn a complex new application
  • Tie in my mature and complex email best practices
  • Develop a new lead nurturing automation campaign far enough to evaluate the product

I felt it was reasonable that my “Aha” moments for Drip would include:

  1. Verifying it could do what my existing provider, Mailchimp, was already doing for my business.
  2. Verify that it could so something that Mailchimp couldn’t via its increased automation features. After all, Drip was going to be more expensive–it should show me some magic relative to Mailchimp during the trial.

As I documented in my write up, I was unable to accomplish these tasks within 14-days despite trying like crazy to get them done.  I had a mixture of problems ranging from product bugs to unclear UX to my own stupid noobie user mistakes.  I could not even get my email newsletter out, despite trying hard for 2 weeks running, so I couldn’t even verify Drip worked as well as Mailchimp, let alone see the impact of its new features.

I wound up sending Drip’s Founder an offer–extend the free trial and work with me until we can make my Drip experience a happy one.  In exchange, I’d buy the product and write about my experiences in places like this blog.  He declined, saying many of his competitors didn’t offer a free trial at all.

Here’s the thing:

If you’re going to offer a free trial, you really should make sure it is long enough that your users can reach the “Aha” moment where they’ve confidently demonstrated your product’s value and it’s an easy choice to reach into the pocketbook and become a paying customer.  Ignore all the other rules of them because reaching the “Aha” moment is the only thing that matters for your Free Trial.  That is its singular purpose.

If you’re not going to do that, why have a free trial at all?  I can’t imagine a reason unless it’s just part of the old bait and switch–get them to commit a little, even just give us their email, and each thing they give up will make the next thing that much easier.  That’s a well-understood marketing concept, and it even works to an extent, but is that really the way to build your successful business?

I can’t believe marketers think so, at least not the good marketers.  Please tell me you’re not in that camp.

Length of trial is something that should be tested, preferably AB tested if you can arrange it. Don’t get too greedy and eliminate your trials before your customers can experience the “Aha” moment that guarantees they will love the product. If you can make that happen in 14 days, great, but don’t just assume that’s the case.  Give them whatever time they need. Even offer to extend the free trial for ANOTHER 30 days if they’re not done evaluating.

You’d be surprised what treating your customers as human beings rather than inventory will do for you.

Posted in saas | Leave a Comment »

Drip: Great Idea, Not Ready for Prime Time

Posted by Bob Warfield on February 5, 2016

As I’ve written recently, I’ve had some problems with my Email provider Mailchimp.  I use Mailchimp to do a variety of emailings as part of my software company, CNCCookbook.  We’re a bootstrapped company that makes software for CNC Manufacturing, and we’ve managed to do very well largely using Content or Inbound marketing.  The company is completely bootstrapped, yet we have traffic that makes us the largest CNC-related Blog and Content resource on the Internet.

The problems with Mailchimp were not life-threatening.  Basically, they were getting some links wrong in the Plain Text version of my RSS Newsletter.  They have been quick to follow up, comping me with some free months and promising to get the problems fixed.  There’s a reason they’re as large as they are and they know how to handle customers.

The thing is, I’ve wanted to move up to a more powerful Marketing Automation solution. To do so, I needed features that were missing from Mailchimp and that it doesn’t look to me like they will be adding very soon.  In essence, they boil down to more powerful Workflows that let me do highly personalized Lead Nurturing.  I believe Lead Nurturing is the next step in getting the maximum value out of my large mailing list (nearly 50,000 members).

So I took this as an opportunity to try another vendor, and I settled on Drip.  The information on their web site made it look like they had the functionality I needed and I had seen that some of the sites I value for marketing information were using Drip.  The price was reasonable for a company like mine–a bit more than Mailchimp but bringing more functionality.  Best of all, I really felt their marketing slogan was perfect for my needs:

Lightweight Marketing Automation That Doesn’t Suck

Unfortunately, despite a week of working hard with Drip, it became clear that it just wasn’t ready for Prime Time.  At least not for a firm the size of CNCCookbook (which doesn’t seem all that large to me being bootstrapped by one guy who is an engineer and not a Marketing Guy).

Let me describe what I was trying to do and what problems I encountered so that others may understand.  By all means, if you’re aware of a solution that can deal with these things without breaking my bank, let me know about that too.  The Marketo’s, Eloqua’s, and Pardot’s in the world can probably do it with ease, but they’re far too expensive.  Even Infusionsoft looks extremely expensive to me.

Step 1:  An RSS Email Newsletter

CNCCookbook has grown through content marketing and I put out 3-5 new articles every week on our blog.  I build the mailing list for that blog via various forms and popups on the web site coupled with premium content offers for signing up.  One of the reasons I picked Mailchimp at the time was that it made it extremely easy to automate a newsletter with an RSS feed, and one of the reasons I picked Drip is that it clearly advertised the same capability.

Drip does a lot of things with their RSS (and other email workflows) that I love:

  • With one click you can specify to do a follow-up remailing to those who didn’t open the first one.  This is hugely valuable all by itself.
  • Their email creation UI is fully on part with Mailchimp’s and even a bit cleaner and easier to use.

But, there were problems–some major, some minor, all added up to my not getting a single Email Newsletter out before I decided to cancel my trial after a little over a week of intensive effort.  Here’s what I found:

  • You have to request the RSS Feed feature be turned on.  I found this to be odd and off-putting.  It’s like they’re not very proud of it or something.
  • It does show up in the UI looking like something of an after thought.
  • There’s no way to do Mailchimp’s useful “Forward to a Friend” link.
  • You can’t customize the Subscription Management page.
  • You can’t manually control whether a user gets the HTML or Plaintext version.  In fact, I don’t think you have a way to even tell which one a given user has chosen though the UI shows both.
  • I was never able to successful send myself a Plain Text version so I could verify it was good to go.
  • Importing my Mailchimp list took hours.  Makes me wonder just how scalable this SaaS app is in an age of Cloud Scalability.

That was all stuff I got my head around and was willing to move forward with.  But then there were some major gotchas.  For example, you can set the RSS up to generate the bulk mailing but wait for you to approve it before it goes out.  You can even trigger its generation without waiting for the once a week date so you can use it to debug your efforts.

Bravo, very cool feature!

But the bad news is, each time you trigger it, it won’t run again until the specified interval.  So, if you test it, but don’t send the mails, you can’t do it again for real for 1 week.  Whoa, totally unworkable and the reason I never sent one email newsletter.

Lead Nurturing and Fancy Workflows

This is where the product really shows promise.  On paper, at least, it is capable of much Marketing Automation Coolness.  Want to trigger actions based on what people are doing on your site?  This is the Holy Grail of email personalization, and Drip can do that.  You drop a little Javascript snippet on every page and voila!  They are now monitoring all that activity.  You can even bring up a subscriber and see the activity.  Tres Cool!

Want to know if they opened emails or clicked through?  No worries.  There’s even a lead scoring mechanism.  Oh boys!  Now you’re ready to put together some awesome Lead Nurturing Workflows, right?

Well, not so fast.  Let me describe the very basic lead nurturing program I came up with, and my attempts to implement it.  Here is the basic Lead Funnel I was after:


No Rocket Science, right?  The Brand Loyalty stage is about our giving value in the initial emails by sharing our most popular and valuable articles.  Gradually, we start to introduce some popular articles that are about the sorts of problems our software addresses.  Then we provide articles that show how our software solves the problems.  Finally, we provide articles that show why we are the best choice.

What we want to use the workflow in a product like Drip to do is to determine which articles are being read.  Based on that, we may escalate or fall back from one stage in the funnel to another:


Based on which email links readers click, we may escalate them to later funnel stages.  If they quit clicking or opening emails, we drop back and start over because they’re not ready…

Again, this is pretty basic lead nurturing, so I’d expect most products to be able to handle this kind of thing.  Here are the obstacles I encountered with Drip:

  • Inability to work with large numbers of links in Visual Workflows.  For example, they’re very excited about their new Visual Workflow Editing.  It looks awesome in the demo, but for even middling complex workflows it is very cumbersome.  For example, you can’t horizontally scroll the diagrams.  I was stretching the window across 2 32″ monitors but still lost the ability to edit when I wanted rules based on 5 links.
  • You can use their Rule Editor, but it is going to be a lot of work.  What would be ideal would be to simply enter a list of links that trigger a new campaign, with one campaign assigned to each funnel level.
  • There’s one single lead score and one threshold for everything.  I need separate funnels, campaigns, and lead scores for each product.  I want to potentially trigger transition to another level of the funnel not just on links clicked but on leadscore.  Maybe a score of 25 = Awareness, 50 = Consideration, and 75 = Decision, or some such.  Not possible–Drip’s Lead Scoring is way too embryonic.  It would also be nice to be able to reset or reduce the lead score if the prospect fails to move forward and we fall back to wait for another time.

There were a number of other detail level fit and finish issues, but I tend to overlook those if a company is moving at a good clip and working with me.  Speaking of working with me, I will say that Drip has some of the best Customer Service I’ve yet come across.

What Now?

I really wanted to work with these guys, their product has a lot of promise.  But I had so many problems during the 21 day trial it was clear I wasn’t going to get it figured out.  So I sent the founder a proposal.  If he’d comp me a quarter and work with me on the issues, I’d work with him.  I’d write about his product, serve as a case study, and provide him with input.  In the end, I’d be a decent sized account for him too as we’re off the top of his published plans.

I was surprised when he turned me down:

Hi Bob,

Thank you for taking the time to put your thoughts down and let us know your situation. I’m sure it’s been frustrating so far as you’ve attempted to get setup, and I appreciate you touching base about this.

From your email, it sounds like a tool like MailChimp, AWeber, or ConvertKit is actually going to be your best bet. It seems that Drip isn’t a fit for what you’re looking to do based on the number of issues you’ve faced. We are unable to extend trials past 21 days as you’ve requested (our competition, such as Infusionsoft, AWeber, ConvertKit, do not offer free trials at all).

With that said, I do appreciate you getting in touch and I’m sorry this last week has been a challenge. I wish you the best of luck with whichever provider you settle on.



Rob Walling


I probably shouldn’t have been, but I have always gone out of my way to work with folks who are providing good feedback about problems that I knew we would have to solve to move forward.  CNCCookbook seems to me is small enough and the problems we had seem broadly applicable enough I would’ve thought we were in that category.

In any event, we have parted company.  I wish Drip all the luck, as I mentioned, I really believe in their core value proposition.  Companys like CNCCookbook need affordable marketing automation.

I do have a couple of other potential solutions in mind.  Heck, maybe Mailchimp will keep moving in this direction, I don’t know.  I will keep you posted.


Posted in saas | 7 Comments »

There’s Something Terribly Wrong With Mailchimp

Posted by Bob Warfield on January 29, 2016


Unfortunately for me and my customers, my campaign went to 50,000 and it was wrong due to Mailchimp bugs…

I use Mailchimp as my email provider for my business, @CNCCookbook.  For the past couple of weeks there’s been a terrible bug.  My email RSS newsletter come out with all bad links for users that selected the Plain Text version.  Every single link is bad the emails–they either go to a non-existent page or to a totally unrelated page.

At first, Mailchimp tried to convince me it was a format problem at my end because I was using the matching quotes so many editors produce.  The problem is, you don’t have to look at the XML for the RSS feed very hard to see that explanation is ridiculous.  Sure, those quotes are there, but the syntax of the RSS Feed makes it very obvious that those quotes are part of the article and not part of the link.

After two weeks of back and forth, they finally admitted they had a problem and that this was, “Not expected behavior.”  They weren’t able to quite bring themselves to use the “B” word–BUG, but that’s clearly what it was.  They also informed me they had no idea how long it would take to fix.

I waited a week, my newsletter went out this week, and I got more complaints from customers.  One was particularly galling–my customer, whose is male, was addressed as “Lillian” in the newsletter.  I quickly checked their profile in Mailchimp and it was not “Lillian”.  In fact, there are no Lillians in the mailing list whatsoever.

At this point, Mailchimp has been slow to admit there is a problem, slow to get a fix (still don’t have one) and it’s a serious problem.  I found another similar report on Twitter, so I’m not the only one.  After the first week of trying to get Support to even acknowledge the problem, I took to copying the company’s founders on the emails.  I even asked my own audience to write to them in my blog post explaining what had happened.

Mailchimp doesn’t really respond much to all this.  There is no sense of urgency.  If the problem is widespread, it’s a real disaster for them.  From my own perspective, the poor customer service (it’s just totally unresponsive) and the fact that this isn’t the first time I’ve had customer service issues with them have led me to start looking for a new provider.

Posted in saas | 5 Comments »

It’s Time to Tax the Robots, Not the People

Posted by Bob Warfield on May 20, 2015

I read a fascinating piece on the economic impact self-driving cars and trucks will have and it’s not a pretty sight.  A quick glance at this map showing the most common job in each state makes it clear:


The most common job in each state in 2014…

You don’t have to study the map very long to conclude automating truck driving away as a way of life is going to have a profound effect on our economy.  When you consider that truck drivers make an average of $40,000 a year, which is more than almost half of all tax payers, and you add in all the jobs related to the truck driving industry, it may be one of the biggest impacts we see in our lifetimes.

Can this really happen?  When will it happen?

The article goes on to show that the world’s first driverless truck has already hit the road in Nevada, and was built by Mercedes Benz.  They present estimates from a variety of sources that suggest somewhere between 2020 and 2030 completely autonomous trucks will begin to take over by storm.  Google has demonstrated self-driving vehicles and Teslas have the capability already built in to cars already on the road.

All of the proponents of driverless vehicles are loudly trumpeting that the vehicles are safer, cheaper, and more fuel efficient.  But what they’re not considering is the impact on the economy.  What will all the truck drivers that lose their jobs do to earn a living?  Who will benefit?

As it stands, large organizations that can afford to buy driverless vehicles will be the beneficiaries and nobody has a plan for what will happen to the truck drivers.

This looks like the third tranche of job-destroying disruption.  The other two–the destruction of Retail jobs first by firms like Walmart and later by the Internet, and the Offshoring and Automation of Manufacturing jobs, are well underway.  The economy is still limping in the wake of the Great Recession, and it’s tough to say we’re out of the woods except for the most dyed in the wool political supporters who want to claim victory for their side.  Meanwhile, Main Street America is braced and wondering what the next big shock to the Middle Class will be.

The article argues its time to get some sort of Basic Income plan in place to provide a Safety Net.  Safety Nets are fine, but I want to know how the Middle Class can do better than a Safety Net.  A vibrant Middle Class does not sit at home waiting for its next Government Safety Net check to do something.  A vibrant Middle Class has hopes, dreams, and upward mobility. Those are all things a Safety Net can’t cover–only Opportunity can provide hope or fulfill dreams.

It’s time to start Taxing the Robots, Not the People.

If the Robots are going to take over more and more of the economy to the detriment of the People and the benefit of those few who can own thousands of Robots, why not tax the Robots?  In fact, why not look at any practice that wholesale destroys lots of jobs as being worthy of taxes to pay for programs to help those who’ve been displaced?

If we look at it that way, there are several areas to think about applying progressive taxes:

–  Taxes on automation and robots

–  Taxes on offshoring jobs

–  Taxes on monopolies that take over markets and then use their unfair influence to gut jobs by destroying all competition

These taxes will automatically be progressive.  They will help balance the playing field so that progress can still come, but there are costs that result in funds to help those displaced by progress.  We don’t want to eliminate the progress, we just want to even out some of the unfairness that comes when we let progress run completely roughshod.

So far, we’ve done very much the opposite, which is part of our problem.  Our system makes hiring more expensive not less.  However good its intentions, Obamacare is a net job reducer because businesses have to pay for it based on how many employees they have.  What if instead they paid based on how many robots, or on how many jobs they had offshored?

There’s been such a massive transfer of wealth that clearly there is money to pay for such programs.  Much more than enough.  While we’re at it, we should be exempting smaller businesses from such programs.  It’s been well proven that Small Businesses are the engines of job creation and growth.  As a nation, we’re sold on the idea of more and more progressive taxes for people, but we have left off progressive taxes for businesses.

We have a tax system that allows 43% of Americans to pay no Federal Income Tax.  Why not a system that allows the smallest 43% of businesses to pay no taxes?  That would dramatically level the playing field and re-ignite Small Business growth.

So, in a nutshell, what we could do to offset a continuing economic disaster for the Middle Class would be:

1.  Tax Robots, Offshoring, and Monopolies so that organizations involved with these practices have the highest tax rates

2.  Radically lower taxes for Small Business.  Some meaningful fraction of them shouldn’t have to pay taxes at all.  Perhaps not 43%, but certainly the 20% or so smallest business.  Make up that lost revenue by increased taxes on larger businesses.

3.  Make #1 and #2 net positive tax revenue generators to allow for new programs.

4.  Put a solid Safety Net in place.

5.  Stimulate the Small Business economy with what’s left.  In addition to Radically Lower Small Business Taxes, we should increase the availability of Small Business Loans and we should make Education cheaper.  The latter will make it easier for those whose jobs were automated away to retrain as well as ensuring an increasing pool of talented labor.

The biggest obstacle in all this thinking is that currently, the people calling the shots in terms of lobbying and poltiical contributions are precisely the ones we propose to have pay for these new programs with new taxes.

How will we ever break out of that cycle?

Posted in saas | 2 Comments »

Oh Dear, the Green Pundits Don’t Understand the Cloud or Multitenancy

Posted by Bob Warfield on January 16, 2015

forestRecently I was drawn into a discussion of how Green the Cloud is where I responded as follows:

SaaS is going to come out ahead of any reasonably calculation of carbon emissions versus on-prem. Multi-tenancy is just a lot more efficient. Look at the data centers of companies like Google, Amazon, and Facebook. Most corporates wish they could come close as they watch these companies dictate every detail right down to the exact design of servers in order to minimize their costs. As everyone agrees, most of that cost is energy.

So choose SaaS if you’re worried about carbon, and yes, it could become another axis of competition in terms of exactly which Cloud provider does it best.

Tom Raftery immediately responded:

The answer is that it depends, tbh. It depends entirely on the carbon intensity of the data centre (where it sources its energy), not the efficiency of the data centre.

If you have a data centre with a PUE of 1.2, and it is 50% coal powered (not atypical in North America, Germany, Poland, and others, for example), it will have a higher CO2 footprint than a data centre with a PUE of 3.0 powered primarily by renewables – again I have run the numbers on this and published them.

Similarly with on-prem. If I have an app that I’m running in-house, and I’m based in a country like Spain, France, Denmark, or any other country with where the electricity has a low carbon intensity; then moving to the cloud would likely increase the CO2 footprint of my application. Especially if the cloud provider is based in the US which has 45% of its electricity generated from coal.

Tom is the chief analyst for Greenmonk, which writes about this sort of thing for a living.  He’s been quoted by others who are in the same camp such as Brian Proffitt on ReadWriteWeb.  And who wouldn’t love a nice juicy story to put those darned Cloud vendors in their place?  Those guys have been riding high for too long and ought to be brought down a notch or two, harrumph.

I have a lot of problems with this kind of math–it just doesn’t tell the whole story.

First, I can’t imagine why Tom wants to be on record as saying that PUE (Power Usage Efficiency) just doesn’t matter.  Sure, he has found some examples where CO2 footprint overwhelmed PUE, but to say the answer depends entirely (his word) on the sources of the data center’s energy and not on the efficiency of the data center just seems silly to me.  Are there no data centers anywhere in the world at all where PUE matters?  Did all the Cloud data centers with great PUE just magically get situated where the carbon footprints are lousy enough that PUE can’t matter?

I’m very skeptical that could be the case.  You must consider both PUE and CO2 per Kilowatt Hour, how could we not when we’re talking per Kilowatt hour and PUE determines how many Kilowatts are required?

Here’s another one to think about.  If this whole PUE/CO2 thing matters enough to affect the economics of a Cloud Vendor, we should expect them to build data centers in regions with better CO2 energy.  Since they put up new data centers constantly, that’s not going to take them very long at all.  Some are talking about adding solar to existing facilities as well.  Now, do we want to lay odds that corporate data centers are going to be rebuilt and applications transferred as quickly for the same reasons?  If you’re running corporate IT and you have a choice of selecting a Cloud Data Center with better numbers or building out a new data center yourself, which one will get you the results faster?  And remember, once we are comparing Apples to Apples on CO2, those Cloud vendors’ unnaturally low PUE’s are going to start to haunt you even more as they run with fewer Kilowatt Hours.

Multitenancy Trumps all this PUE and CO2 Talk

But there’s a bigger problem here in that all data centers are not equal in another much more important way than either PUE or fuel source CO2 footprints.  That problem is multitenancy.  In fact, what we really want to know is CO2 emissions per seat served–that’s the solution everyone is buying.  Data centers get built in order to deliver seats of some application or another, they’re a means to an end, and delivering seats is that end.  The capacity they need to have, the number and type of servers, and hence the ultimate kilowatts consumed and carbon footprint produced is a function of seats.  Anyone looking purely at data centers and not seats served is not seeing the whole picture.  After all, if I run a corporation that has a datacenter, it’s fair to charge the carbon from that datacenter against my corporation.  But if I am subscribing to some number of seats of some Cloud application, I should only be charged the carbon footprint needed to deliver just those seats.  Why would I pay the carbon footprint needed to deliver seats to unrelated organizations?  I wouldn’t.

Corporate data centers have been doing better over time with virtualization at being more efficient.  They get a lot more seats onto a server than they used to.  The days of having a separate collection of hardware for each app are gone except for the very most intensive apps.  But that efficiency pales in comparison to true multitenancy.  If you wonder why, read my signature article about it.  I’ll run it down quickly here too.

Consider using virtual machines to run 10 apps.  Through the magic of the VM, we can install 10 copies of the OS, 10 copies of the Database Server, and 10 copies of the app.  Voila, we can run it all on one machine instead of 10.  That’s pretty cool!  Now what does Multitenancy do that the VM’s have to compete with?  Let’s try an example where we’re trying to host the same software for 10 companies using VM’s.  We do as mentioned and install the 10 copies of each kind of software and now we can host 10 tenants.  But, with multitenancy, we install 1 copy of the OS, 1 copy of the Database, and 1 copy of the app.  Then we run all 10 users in the same app.  In fact, with the savings we get from not having to run all the VM’s, we can actually hose more like 1000 tenants versus 10.

But it gets better.  With the Virtual Machine solution, we will need to make sure each VM has enough resources to support the peak usage loads that will be encountered.  There’s not really a great way to “flex” our usage.  With Multitenancy, we need to have a machine that supports the peak loads of the tenants at any moment in time on the system.  We can chose to bring capacity on and off line at will, and in fact, that’s our business.  For a given and very large number of seats, larger than most any single corporate application for most corporations, would we rather bet the corporation can be more efficient with on-prem software in its wholly owned data center or that the SaaS vendor will pull off far greater efficiency given that its software is purpose-built to do so?  My bet is on the SaaS vendor, and not by a little, but by a lot.  The SaaS vendor will beat the corporate by a minimum of 10-20x and more likely 100x on this metric.  You only have to look to the financials of a SaaS company to see this.  Their cost to deliver service is a very small part of their overall expenses yet most SaaS apps represent a considerable savings over the cost of On-Prem even though they carry the cost of delivering the service which the On-Prem vendor does not.


Raftery says, “energy use” and “emissions produced” have been conflated to mean the same thing.  I say he’s absolutely right about that but hasn’t seen the bigger picture that it is not energy use nor emissions produced in isolation, it’s seats delivered per emissions produced.  Itt’s having the flexibility to make a difference rapidly.  And that is why we bet on the Cloud when it comes to being Green.

Posted in cloud, data center, enterprise software, saas, strategy | 1 Comment »

What Can a Poor Dumb Engineer Do That Most Marketers Can’t?

Posted by Bob Warfield on January 12, 2015

This is a tale of bootstrapping and bucking the conventional wisdom.  This is a tale of applying an Engineer’s overly top down facts sifted through logic lens to what is traditionally a touchy feely shoot from the gut discipline.  This is the story of how my little one man bootstrapped SaaS company competes with giants in marketing, even though I am but a poor dumb engineer.  And yes, the title is pure link bait.  Many marketers can do and do-do (uncomfortable with that) this sort of thing.

My company is called “CNCCookbook.”  We are a niche SaaS software company focused on CNC (Computer Controlled) Manufacturing.  Both the CNC and SaaS within the CNC world are very much niche plays.  Public companies in manufacturing software are few and far between, and most VC’s wouldn’t touch it with a ten foot pole.  To paraphrase a more colorful turn of phrase, they wouldn’t invest in it with YOUR money, let alone their own.  Most of the software here is not SaaS–it arrives on a bunch of CD’s and is after much installing, dongling, and license keying, you’re left with a vintage 90’s User Experience.  I love this niche because what I know from working in more progressive markets gives me an edge that I bring to Manufacturing software.  I love it also because it is my hobby–yes, Virginia, I use CNC machines to make things for fun.

I’ve been at it in this marketplace for a while now.  I grew the business to the point where it pays me as much as I have made at any Silicon Valley Startup except those where stock options mattered.  I should know, CNCCookbook is my 8th trip unto the breach, dear friends.  Now what’s all this about competing with giants in marketing?

Let’s start by looking at how CNCCookbook is marketing.  I came across this great infographic from Marketing Tech Blog that shows the conversion rates of various B2B marketing techniques:


B2B Sales Channels via Marketing Tech Blog

Strictly speaking, CNCCookbook is both B2B and B2C since the hobby CNC market are decidedly “C’s” while the business CNC market are “B’s.”  No matter.  The takeaway from the article is that a lot of the marketing world is a bit surprised at how some of the tried and true techniques are not working so well any more.  If we look at what CNCCookbook does from this list, we have tried Webinars and Paid Search and found both to be unprofitable and ineffective.  Today, we’re totally focused on our Website/Content Marketing and Facebook/Twitter.  You could call it luck that we wound up on two of the highest converting channels, but we did test an awful lot of different possibilities and I have had the luxury of seeing how things worked for other companies.  The latter meant that I was unlikely to try event marketing, trade shows, partners, or sales cold calling (it’s just me in this company, I don’t have time to cold call), for example.

The Facebook and Twitter work is almost 100% automated–I simply co-announce any new blog posts to those channels automatically with a WordPress plugin and I make sure any non-blog content I publish is also announced in a blog post.  Simple, and effective.  I monitor these and many other channels for opportunities to engage with my customers, but the rest is automated.

What do I spend most of my marketing time doing?

The answer is writing articles for the blog and web site.  I alternate between blog posts and what I call “Cookbooks.”  Cookbooks are in-depth go-to resources that consist of many articles.  I try to introduce a new Cookbook every year and we’re up to 5 of them at this stage. They’re solid traffic producers and what the marketers like to call “Evergreen Content.”  Good reference content seldom goes out of style and accretes links very well.

And how much time is spent on marketing?  My days and weeks divide roughly into 20-40% marketing with the rest focused on software development.  Since I believe Good Customer Service is critical to marketing, I charge my Customer Service time to Marketing as well.  FWIW, I work 7 days a week and often a fair number of hours, but the business is such that I can do the work almost anywhere–from a Cruise Ship Cabin or a leased Condo on an Exotic Caribbean Island (both are real examples).  As long as I have a decent but not spectacular Internet connection, I get by.

That’s a bit about how it’s done, now what’s been accomplished?  What can one poor dumb software engineer working part-time (20-40% time) accomplish by way of marketing?

There are a lot of ways to look at it.  Ideally, one should look at leads, but it’s hard to find out how many leads the other organizations you want to benchmark against are getting.  As a reasonable proxy, I like to use SEO Traffic.  Who doesn’t love free (heh, we won’t count all those hours writing content!) over the transom leads that come from searchers finding your opportunity for the first time via Google or some other search engine?

To do this comparison, I like to use SEMRush.  It’s been a valuable tool for me in a lot of ways (keyword research, understanding pay per click which I ultimately have given up on, etc.).  You can get quite a lot from it for free even.  Here are a set of comparative SEO stats drawn from SEMRush that compare CNCCookbook’s traffic to others in our industry:


CNCCookbook vs Industry Comps for SEO Traffic…

Pardon the length of the list, but I do this for my own analysis purposes and didn’t want a cherry-picked list.  These are real companies that are big names or names of interest to me in the CNC Manufacturing world.  Some will be familiar like Autodesk, but most are probably unknowns unless you happen to be familiar with this market.  We can pick out some highlights though:

–  Makezine and Hackaday reflect the popularity of the Maker Movement, and so I wanted them here to gauge my impact on that world.

–  Practical Machinist and CNCZone are the two biggest online communities in the space, so were obviously of interest.  Amazing to see CNCCookbook is doing better than CNCZone as it is quite a big community.

–  Haas Automation is the world’s largest maker of CNC Machines and has in excess of $900 million a year in revenue.

–  IMTS is the industry’s largest trade show and an online magazine as is mmsonline.

–  Kennametal, which ranks immediately below CNCCookbook is a $3 Billion a year company that makes cutters and other tooling for CNC.  Iscar is a tooling company that Warren Buffet paid $2 Billion for not that long ago, and I can assure you that tooling companies don’t have zillion to one valuation ratios nor would Warren Buffet buy one if it did.

The point is there are a lot of solid businesses on the list.  Many are quite large and no doubt have large marketing staffs and budgets.  Yet CNCCookbook has been able to make its mark by joining their ranks, at least in terms of search engine traffic.  What a triumph for content marketing–if ever there was a reason to believe in it, seeing a guy work part time to deliver these kind of results ought to be it.

Here’s another list, this time with names that will be more familiar to Smoothspan Blog readers:


Startups, Tech Marketing, and other Tech Related Comps…

These are startups, tech marketing companies, software companies, and other tech related startups I wanted to compare to.  A lot of these are talked about a lot.  A lot of them are in the business of telling others how to do marketing (sometimes I wonder if I should be in that business too, lol).  Considering the amount of resources available to these companies and the fact that as many of them are marketing companies selling to marketers who therefore should know a lot about how to do this, I feel good about how CNC Cookbook ranks alongside.

What Should We Conclude From This?

There’s only one takeaway I would encourage you to have: engineers with limited resources can successfully market their bootstrapped companies.  There are broader ramifications perhaps for broader markets and audiences, but I’ll let others decide whether they want to jump to those conclusions.  I know what I did for my market and what results that produced and I’m very happy with them.  We live in a time when this sort of thing is much more achievable than it once was because the Internet is so much more ubiquitous.  For those who want to follow a similar path I would urge you to choose your market carefully and find one where content marketing (also called inbound marketing) can be extremely effective.  You don’t need to be the world’s great graphic designer (I am so not that!), but you need to be a good writer, you need to have passion for your subject matter, and you need a market whose audience never gets enough content to satisfy their thirst for more knowledge.  If you can put all that together, it’s hard to see how you wouldn’t build a following.

Posted in bootstrapping, business, Marketing, strategy | 3 Comments »

Yes, In It’s Pursuit of Being a High-Margin Luxury Brand, Apple Must Eventually Be Less Functional

Posted by Bob Warfield on November 3, 2014

DSCN0023Even Seth Godin flushes out the Apple Fan Boys sometimes.  David Terrar has an, “I disagree with Seth Godin,” post going.  Seth’s premise, as set forth in, “Decoding Apple as a luxury tools company,” is that eventually, as a company builds a luxury brand, they must choose between luxury and utility or be trumped by another luxury brand that did make the choice.

I’ll cut to the chase–Seth is right and David is wrong to disagree.  Now I’ll explain.

David Gives The Best Counter-Examples to His Own Position

He talks about Patagonia being able to continue on utility and still function as a luxury brand.  But that misses the nuance of Godin’s article.  Patagonia is trumped by Louis Vuitton and countless others precisely because they were focused entirely on luxury with no need for any utility-seeking compromises.

He askes whether Ferrari, Bentley, Bugatti, or Aston Martin compromise their engineering in the interests of luxury, and suggests that of course they do not.  Au contrare, David, aucontrare.  At the risk of mobilizing both the legions of Apple Fan Boys and still more fans of these fine marques, they do compromise engineering.  Enzo Ferrari used to purposefully limit the performance of his street vehicles because he didn’t feel his customers were competent to drive cars with greater potential.

This practice didn’t stop when the Old Man checked out.  Every Ferrari I’ve owned or driven compromised all sorts of handling performance in the interest of greater comfort.  They love that light steering feel and the cars are ever so much more supple over bumps because of course we need to protect the delicate posteriors of our customers.  Luxury?  Absolutely.

Much the same can be said of Bentley, Bugatti, and yes, Aston Martin.  Their cache is exclusivity or outrageousness, not fine engineering.  We may as well lump Lamborghini into all that, if need be.  McClaren?  The jury is still out on that one for me, perhaps there is one shining exception.  Or indeed, perhaps individual models (F40) escape.  But overall, these auto brands are great examples.  Mercedes and Porsche are luxurious, but they are not apex luxury like these other cars.  They have much better engineering and bring more innovations to the table.  Heck, the lowly Corvettes have Utility advantages over many of these cars and are at best low-end luxury.

Watches?  Don’t get me started.  The luxury time pieces favor mechanical movements.  My Rolex loses time every day compared to a decent Swatch.  But it just wouldn’t be right to stick a quartz movement into a luxury watch like the Rolex, Patek, or name-your-expensive-Swiss-timepiece-here.  So they don’t, and have therefore done something to benefit their luxury status to the detriment of their utility functionality.  You could argue they’ve abandoned Utility altogether from any objective measure of what the Utility of a wrist watch ought to be.

Must Apple Do This?  Has Apple Already Done This?

So far, I think most agree that while Apple may have flirted with sacrificing Utility for Luxury, they haven’t actually crossed that line decisively.  There are certainly needless dogmatic issues that drive anyone transitioning from a PC crazy.  I have True Apple Fan Boy friends that only use their Macs with PC keyboards and PC mice.  It seems they want real <Del> keys, <Home>, <End>, scrolling mouse wheels, more buttons and all that jazz that Apple mysteriously refuses to give them.  In the past there have been missing fans and arrow keys that were again nods to Luxury versus Utility.  But Apple backed away from them.  Perhaps they’ll fix their keyboards and mice too.

These things happen because part of being apex luxury is ignoring customers.  After all, you are the infallible arbiters of style.  What you produce is by definition better than what anyone else produces.  What could a mere customer possibly have to tell Louis Vuitton about how to make more luxurious cases?

Yes, it’s arbitrary, dogmatic, and a chase your tail kind of tautology.  But that’s the nature of Luxury and Style versus Utility and Function.  It starts with how the brand is judged, and it would be hard to argue that Apple doesn’t see themselves firmly in the Luxury versus Utility side of the judgement game.  They don’t consider anyone worthy of judging their products.

What About the Ecosystem?

We shouldn’t leave this whole Luxury/Utility dichotomy without touching on the issue of ecosystems.  Luxury goods don’t have them.  Utility goods do.  There can be little question that ecosystems add utility value, though they can also add complexity and other pain.  My Macs plug and play with, well whatever they’re willing to plug and play with.  My PC’s plug into vastly more things but very often they don’t play very well once plugged in.

Apple has sharply limited their ecosystem to that which they strictly control.  In so doing, they have made some choices not unlike the Godin quote:

When Apple dumbs down Pages or Keynote or allows open bugs to fester for months or years, they’re taking the luxury path at the expense of the tools path.

Consider the whole approval cycle for their App Store ecosystem.  They will argue it is there to protect users from junk apps.  I’d argue it’s more about closing Apple’s ecosystem so it cannot threaten them in any way and so they can maximize their profits from it.  Here’s what I mean about the Godin quote being close to home:

One of the great advantages of SaaS is keeping everyone on the same release and making sure that release can be quickly and easily upgraded.  I surveyed Enterprise Software Customer Service one time and discovered that companies estimated that 40-70% of bugs in open trouble tickets were fixed in the current release.  That’s how important updates are–they literally keep 40-70% of your customers from even seeing the bug in the first place.

Yet, because we have to protect these  Walled Garden, it may take quite a lot of time and effort to get a new release approved.

To David’s list of brands and the power of ecosystems, I’ll bring in the world of audio.  Bang and Olufsen are beautiful.  They’re luxury.  But are they utility?  Are they the best sounding audio?  They’re all in one and they eschew an ecosystem, but is that really a good thing if you want to maximize their Utility (e.g. Audio Performance)?  No, not really.  And look at how similar B&O are to Apple.  Interesting parallels there.

Is Apple helping their Utility or their Luxury by limiting their ecosystem so much?  At one time, I think they were helping their Utility, but that balance seems to me is shifting more to Luxury.

Godin’s Decoding of Luxury vs Tools is Not Unlike Michael Porter’s Competitive Strategy

I found Godin’s view of the Luxury vs Tools markets to be not unlike Michael Porter’s venerable Competitive Strategy, for those who remember their B-School studies.  Porter argues that there are only 3 successful competitive strategies:

1.  Be the Best.  Here he means “having the most utility” vis a vis Godin’s take.

2.  Be the Cheapest.  This is Microsoft and Dell to Apple’s “Be the Best.”

3.  Serve a Niche that #1 and #2 are under-serving.  This is where the most Luxurious winds up.

Porter suggests that companies need to pick just one of these strategies.  Trying to serve more than one divides your resources and will allow a competitor that focuses on just one of the strategies to beat you.

What often happens to brands that start out to be the Best is they can’t sustain that advantage.  That requires too much engineering inspiration and innovation, which is just not reliably schedulable when you need it.  So instead, they start to flirt with design inspiration, which can be scheduled and in fact is more desirable if things change often because what that audience seeks is distinction.  They relish the opportunity that there is new distinction constantly being made available for their delight.

The situation for Apple is sustaining that pace of innovation.  My Microsoft Surface Pro 3 is a good example.  It’s hardware is absolutely up to Apple’s standards in every respect.  Apple at this time has no notebook tablet.  Will they build one?  Or will they insist that since they are Apple, the world must follow their lead and they will not be following anyone else?

Can there be a MacBook Tablet, or will it go the way of the keyboards and mice, wishing they had the PC functions or worse, suffering the indignity of having PC hardware plugged into the pristine Mac User Experience?

Is Apple selling the Best Technology or Silicon Haute Coutoure for the Well-Monied Gadget Set?

Oh and David, you mention you wished there were comments on Seth Godin’s blog?  I sure wish Medium where your article appeared had them too.

Posted in saas | 10 Comments »

Secrets of When and How to Talk to Customers at a Startup

Posted by Bob Warfield on October 9, 2014

elephant-with-blind-menJason Lemkin says forget building wireframe UI’s and start out interviewing 20 customers, because you just won’t understand your customers until you do.  Here’s the gist of why you need 20 interviews before you do anything else:

And you have to do 20.  I know it’s hard to get to 20.  But it’s the right number:

  • You need the First 5 Interviews just to truly understand the white space and the current opportunity.  Yes, you probably think you already understand it.  But you are the vendor, not the purchaser.  You need to understand your prospective app from the purchaser’s perspective, for real.
  • You need the Next 5 Interviews to confirm your pattern recognition.  You learn from the first 5, you confirm in the next 5.
  • You need Interviews 11-20 to Nail Your Pitch and Hone Your Thesis.  Once you truly understand the white space from a buyer’s perspective, and you’ve figured out the nuances and challenges … it’s time to nail your pitch for real.  And by doing this, you’ll also hone your thesis and strategy.   That’s what interviews 11-20 are.  To get real critical feedback on what you’ve learned.  To learn about corner cases that may in fact be critical insertion points for you to win.  To dig in on what is really 10x better, not just 2x or 5x better.

And let me tell you, at least from my experience, don’t expect all 20 to be positive.  Many of My 20 Interviews in both my start-ups were very critical.  Or worse, lukewarm.  Lukewarm is even worse, because it says yeah it’s sort of interesting … but no way I’d buy … and implicitly … your idea is a huge waste of time.  I’d rather get the negative feedback 😉

I get the Steve Jobs thing.  You just have to build it.  You do.  But this is SaaS.  You’re solving a business’ problem.  They don’t know how to solve it, or what you should build.  But they do now how to express their problem.  Acutely, and thoughtfully.

Here’s the funny thing–in some ways I agree with Jason and in others I totally disagree.  It depends really on what it is you’re trying to learn from your 20 interviews.  Jason says you’re trying to understand the white space and the current opportunity and that you’re trying to nail your pitch and hone your thesis.  He’s thinking about it like a Sales Guy, more power to him, someone in your company ought to be.  But there’s more to life and startups than Sales Guys.

Here is what I worry about validating in the early days of any startup:

1.  What is the problem we’re trying to solve for customers?

2.  Is it a real problem?  Do a large portion of customers believe they have this problem?

3.  Will the solution we’ve imagined actually solve that problem?  Do the customers agree that it solves their problem?  Can we charge enough to make a real business for this solution?

4.  Do we have a pitch that communicates we have a real and effective solution quickly?

I see Jason’s 20 interviews as helping to solve #2 and #4, but not really making much impact on #1 and #3.  Moreover, I see #4 as being pretty tactical, unless, of course, you need that finely honed pitch to convince investors.  It’s tactical because we won’t need it until it’s actually time to get customers to deploy beta product.  In other words, we have quite a lot of time to solve it.  #3 is not so tactical.  In fact, if we don’t solve #3 right up front, we could easily spend the bulk of our time building a product that we think solves the customer’s problem but that customers aren’t confident in.

Let’s drop back and work on each one of these 4 critical questions and see how and when in the startup cycle they should be tackled.

1.  What is the problem we’re trying to solve for customers?

This is a tough one for many entrepreneurs.  I’ve seen a few decide to try to find a hard problem to solve by interviewing potential customers.  What keeps you up at night?  What do you hate about your job?

Ugh.  That seems so hit or miss.  None of the ones I met who’ve tried this got very far with it.  They got problems that software was just simply not the cure for or they got problems that are more conditions of the human race than anything.  Jason says customers, “Know how to express their problem.  Acutely, and thoughtfully.”  Actually, they don’t, not so much.  They know how to resonate with a problem that you’ve stated acutely and thoughtfully.  They know how to resonate when you state a problem as a near miss to how they really think about it.  But customers are not product designers nor company founders for the most part.  They’re severely myopic.  As Henry Ford famously put it,

“If I had asked people what they wanted, they would have said faster horses.”

So don’t rely on customers to tell you what the problem is.  Really on them to confirm you’ve found a problem they feel.  Later, they’ll remember it as you having discovered the problem from them, but that’s not really how it works most of the time.

There are a lot of reasons for this.  The myopia is one but another is they just don’t know much about the medium you work in–software.  They have little idea what software can do for them.  They think of most software in terms of software they already have, which is another form of myopia.

You are not going to figure out that problem, in all likelihood, through a few simply interviews.  You’re going to have to live the life of your customers for a while, walk in their shoes, and feel their pain.  You need to know their domain intimately, which is not something that’s going to happen in 20 interviews, no matter how good they are.  That’s how two great Sales Guys wound up creating two great CRM companies–Tom Siebel with Siebel Systems and Mark Benioff with  I interviewed with Tom Siebel when he had less than 20 employees and the one thing that was absolutely clear about the man was that he knew the domain and its problems cold.  That’s the kind of solid domain knowledge you really should have in your startup.  Who can you point to who has lived and breathed the problems and knows them cold?

Is that the only way?

No, there are plenty of exceptions.  There are proxies available too.  Finding a large online community of your desired customer can give you huge insights into what their world is all about.  What do they ask questions about most frequently?  What do they complain about frequently?  These communities are so helpful to startups both for gathering information as well as for getting out the word that I’m not sure I’d want to do a startup that couldn’t identify an online community specializing in its customers.  In this age of Content Marketing, it seems to me that such a community would be a very valuable indicator that the market was going to be reachable and at reasonable costs.  I don’t want to have to advertise or cold call my way into existence, though many have certainly done so.

To put this into the perspective of Jason’s 20 interviews, you need domain knowledge for your startup before anything else.  You won’t get it from the 20 interviews, and it is just table stakes that you have to find.  Maybe you’re counting on a founder for it.  Maybe you have the world’s ultimate advisory board.  Maybe you’ve sold to these poeple in a former life and know all about them.  Maybe you’ve spent a year studying and interacting with their online communities.  Whatever it is, you’d better have it.


2.  Is it a real problem?  Do a large portion of customers believe they have this problem?

Having gotten your domain knowledge together, you believe you’ve discovered a real problem.  Now that you can articulate that problem, it’s time to confirm it with potential customers.  Jason’s 20 interviews are perfect for this stage.  I don’t know that there is anything magical about 20 or that this even has to be done via interviews.  If you’re going to be using feet on the street to move your product (e.g. a scratch golfin’ highly paid salesforce), you should probably get started with interviews.  If your Sales Guy can’t line up 20 interviews in his sleep, there is something wrong, so may as well set him to the test.  If you weren’t planning to get a Sales Guy until later and you can’t line up 20 interviews on your own, you better get the Sales Guy sooner.  There’s a lot of good that comes from achieving the 20 interviews and here Jason and I agree wholeheartedly.

But in addition to 20 interviews, I highly recommend a few other things.

One of my mentors has used the method multiple times of creating a web site that’s all about the proposed problem and solution.  He sets it up like there’s a product ready to go, except there’s no way to order it.  You can simply request more information.  He gauges the quality of the idea by whether he can get many information requests.  And yes, he understands the need to do some tuning up before giving up.  This is a case where having online communities of your potential customers is awesome.  If you have a simulacrum site as described (looks like a real company and product), you can ask the folks in the community what they think of the company and idea.

If you don’t like that approach because it just feels a little too funky, try my approach.  I started a blog before I started my current company and I waited to see if I could drive significant traffic to that blog discussing the kinds of problems I wanted to solve.  I would talk about how people were solving their problems today, rather than how I proposed to solve them.  I measured traffic using all of the standard web analytics to see what resonated and what did not.  I built a lot of credibility and I had a following already in place that I leveraged to a significant Beta test and significant cash sales when I was finally ready to launch the product.  But, before I started building the product, I knew from how people were reacting to my writings that I understood the customers and their problems very well.  I achieved what I call “Content-Audience Fit” (a precursor to Product-Market Fit).  More about that below under #3.


3.  Will the solution we’ve imagined actually solve that problem?  Do the customers agree that it solves their problem?  Can we charge enough to make a real business with this solution?

Here’s where I probably disagree with Jason the most.  He says, “So if you haven’t started yet, as fun as it is to just build the wireframes and get a codin’ … do the 20 Interviews.”

Here’s my problem–as I’ve mentioned, the Customers really can’t articulate their problem unaided.  They can only resonate with your articulation.  We may have to agree to disagree on that, but figuring out how to resonate well is a huge function for Sales precisely because the customer can’t do it for themselves.  They often need help even in how to present it to each other to get buy-in within their organizations.  When you go interview them, you’re going to get a variation on the three blind men encountering an elephant for the first time.  One touches the trunk, and thinks it’s a snake.  Another touched the tale, and thinks it’s like a straw fan swishing back and forth.  While the third touched the legs and pronounced that elephants are like trees.

When it comes to envisioning a solution, especially in software, things get far worse.  At least they have all experienced the problem, even though it may appear to be a snake, a fan, or a tree.  But nobody outside your company even has a glimmering about your proposed solution.

I’ve done lots of focus groups over the years and lots of the kinds of interviews Jason talks about, and I am here to tell you it is pointless to do either if you expect to get feedback about a software solution unless you have at least some wireframes and storyboards to show.  With modern tools, it’s just not that hard to produce these things.  I’m working on our third product at CNCCookbook.  I was able to put together a UI prototype that is substantially what our finished product will be with about 6 weeks worth of effort.  It’s been hugely valuable in securing feedback for the product and I have learned an awful lot from it.  Perhaps the biggest problem it has is people start to mistake it for a finished solution or they assume it’ll be ready much sooner than it will and they get too excited.  They’re ready to buy immediately.

Yes, it may be fun to build wireframes, but it is also fun to have real meaningful conversations with prospective customers about your proposed solution.  You’re going to learn so much more about everything if you can do that around a real demo.  You’ll learn about positioning, sub-problems and edge cases that as Jason says are critical insertion points to win.  You’ll learn whether your proposed solution really fixes the customer’s problem in their eyes.  You’ll feed on the enthusiasm they have for what they see, and that enthusiasm is valuable fuel for your startup, for convincing critical new hires, and for any potential investors you may have.

If it’s really going to be hard for you to get interviews with 20 real prospective customers, people who are solid citizens in their markets and who are not your buddies.  People who will give you the straight scoop, help guide you, and who you’re hoping will be early adopters.  If it’s that important and that tough, I can’t imagine wanting to do it without bringing along a UI mock up.  It isn’t just my current venture, I have done this at every single one of the 7 startups I’ve been with.  I have done it for every new product release.  It’s actually integral to how I approach the Agile Software experience.

If your Sales Guy can’t get 20 meetings, get another one.  If your Product Guy can’t get you a UI prototype quickly, get another one there too.  Both are equally as important.

One more thing–that last part, “Can we charge enough to make a real business with this solution?”  That’s critically important to answer ASAP.  It’s also nearly impossible to get a very good answer to it without a UI prototype.  Yes, they will give you some answers, but I am talking about real answers that will hold water when it’s time to cash the checks. Don’t you want to know whether customers will pay up for what you’re going to build as early as possible?


4.  Do we have a pitch that communicates we have a real and effective solution quickly?

This is what I call “Content-Audience Fit“.  I believe achieving that fit needs to come ahead of finishing the product if for no other reason than that you won’t know if the product is finished nor will you be ready to efficiently leverage content for product traction unless you do.

Jason’s 20 meetings go towards this end, but it’ll take more than 20 to really nail your pitch.  Personally, I don’t like to burn real perspective customers, investors, or other scarce as hen’s teeth resources for a company if I can find another way to test this stuff out and perfect it.  The online world and Content Marketing are your gateway to doing so. Once you can resonate with those audiences, break out the Rolodex (kids you’ll have to look up what that is) and start asking for meetings.  You’ll be bringing to that meeting a laundry list of good-as-gold asssets:

–  By this point you have a UI prototype to demo.

–  You have verified that you can talk about the problem and with the audience with enough credibility that they’re at least starting to come to you for answers.

–  You’ve had the opportunity to test a number of things with your growing audience.  You’ve probably even been able to do some surveys.

–  You have a corpus of content that you can point potential meeting invitees to that helps establish your bona fides and gets the conversation off on the right track.  If done right, this can be a warm call and not a total cold call.

Most importantly, none of this is all that expensive or time consuming.  You can do it on your own nickel without waiting for a Series A VC round.  I know I have more than once.  I’d set a goal of 3-6 months to get to this point.  If everyone is firing on all cylinders, you’re producing good content, you’ve gotten through your UI prototype, and you’ve made contact with a decent sized audience, you’ve accomplished a lot at your startup.  You’re right where you need to be.  Now line up those 20 meetings, get in there and make those 20 people your first Beta tests, and hit the ball out of the park for them.  They’ll love you for it and you’ll have set the stage for your next 6 months as you drive to launching the Beta and eventually real Sales.

Posted in bootstrapping, business, enterprise software, saas, strategy, venture | 1 Comment »


Get every new post delivered to your Inbox.

Join 368 other followers

%d bloggers like this: