Testing 1234 -- Strengths, limitations and opportunities in market testing

When you read through this posting, you'll see a "Test 1234" where you can help out, individually, because the only way I can figure out why our emails aren't working so well is to find if there is a real problem here. Testing can be invasive, irritating, subtle, or (perhaps i will find out) useful in involving your community in your marketing and business challenges. Thanks for helping out. You can click on the image to send the test email or email buckshon@cnrgp.com, but remember to phone or comment to validate the results.

Is your bright marketing idea really so bright? Often, I have to admit, my "best ideas" bomb in the real world, have unintended negative consequences, or simply don't work quite the way I thought they would.

Then of course, there are the ideas that didn't seem to have much going for them in the first place, but for some reason seemed worthy of pursuing. This blog, for example.

Is testing a way around this challenge? Can we keep ourselves out of trouble by trying the idea on a small enough sample and/or short enough time that we can tell if our marketing ideas are good, bad or ugly -- and catch disasters before they happen?

The answer at first may seem to be an obvious "yes", and I'm pretty confident I've saved my skin from some bad idea implementation by trying things out with a simple test first.

In one case, for example, we were exploring starting a new leads service, providing really valuable data in co-operation with a major partner, using relatively low cost print-on-demand technology and providing genuine value and service to readers.

My sixth sense told me to be careful, so I asked a non-employee sales representative (who I know was good at the type of selling required) to work with me for a couple of days to see if the idea would fly. It bombed.

But here is the rub, and challenge, of testing. Maybe the test really wasn't valid. Possibly some variable within the testing process had thrown the numbers off, and with a slight rejigging of the concept, either within framework, market or focus, it would have worked wonderfully. We'll never know, of course.

Conversely, we know of awkward tests that cause real market problems; specifically the infamous "mike tests" where politicians and celebrities are caught with their pants down (or, to use another cliche, their foots in their mouths) when they say things that really aren't meant for broadcast in an AV test.

Finally, in the construction marketing community, we have to contend with the reality that many of our most interesting and important projects arise from long-term and interconnected relationships, which run counter to the principals of simple A/B testing and other measuring methodologies. If the process has a long lead time, how can we figure out when and what to test, and where are the boundaries of good metrics and simply wasting our time counting things that aren't really important?

Here are some thoughts and guidelines that can perhaps help out on our testing methodologies.

If it can be measured, it can be tested.
You will find test results are most meaningful and easy to implement if you have some simple metrics to assess your ideas and results. Online resources provide a wealth of measurement tools, including hits, conversion rates, and the like, and you can develop simple processes to tell if something is working or not. Once you have a baseline, for example, on responses to your emails, you can try different things with A/B samples, sending one message to half the list and another message (with the change you are trying to evaluate) to the other. You'll know which works right away.

Testing works best if is real time, not obtrusive, and doesn't seem to be a test.
The "Hawthorne Effect", where the fact that by letting people know you are testing changes behaviour, is a real risk if you are obvious in your evaluations. So you need to be discreet. Market surveys are often problematic -- many people hate surveys and won't respond (your most important sources of information), or you need to be intrusive with phone calls and personal contacts (thus alienating the person you are seeking to survey, or again, you achieve invalid results.)

You need to catch subtle clues as well as objective measurements in your testing (carefully).
Sometimes the real results are not the ones you think you were going to find; so you need to probe into the observations you obtain and listen carefully to subtle clues. Of course you may have it wrong, but you may also score major insights.

So, yes, test your assumptions, your ideas and your marketing strategies, especially if you are planning to commit major resources to the process. But be wary of the testing pitfalls.

And now for a real-time test (and a request for your assistance) because we are having some strange email problems. Some emails I and my team are sending are not going through; in other cases, people are sending emails to us, which we aren't receiving. The problem may be spam blocks; I don't know.

1. Please send an email marked "Test 1234" to me at buckshon@cnrgp.com.
2. At the same time, please post a comment to this blog (which operates on a different server) or phone me at 888-432-3555 ext 224. (If you wish to leave a comment and remain anonymous, just make reference to something in your email which won't disclose your identity.)

If you complete this test, I'll send you a copy of my new publication: The Art and Science of Publicity, but most importantly, a really big "Thank You".

The more responses the better -- the key measure will be the number of communications we receive through the blog comments and phone messages which fail to correlate with the number of emails we receive. But I'll learn other things, as well, which I'll share with you tomorrow.

Again, simply email a note to me at buckshon@cnrgp.com with a subject line: "Test 1234" and communicate offline, either by a blog comment or phone call to 888-432-3555 ext 224 to tell me you've sent the email. Thanks.

Tricks, gimmicks, and shortcuts

This comment I received overnight (about 2 a.m. EDT) certainly seemed flattering.

Dude, GREAT blog. This whole site that you have set up is top-notch. Well done. I stumbled across it as I was looking for an article on general contractors.

I’m really looking forward to reading all of your archives. Terrific job, keep it up!

However, my sixth sense immediately told me something didn't ring right here. Why would anyone write such flattering remarks for an old posting relating to the Ontario General Contractors' Association? Must be, I ultimately correctly deduced, an effort by someone to steal a back link through a form of comment spam/management.

However, the back link wasn't obvious from the surface. I only discovered the trick when I posted the remarks and discovered the link embedded in the name "Janee Martin". I then immediately deleted the comment, after a cursory examination of the referenced site.

Paradoxically, the site may have had some value to the blog's readers. But why should I extend it any respect if the organization seeking attention is playing tricks and games, and presumably using an offshore service, to manipulate things.

These observations lead to a larger, more challenging question: Is it wrong to use tricks, gimmicks and short-cuts to achieve your marketing aims? And here, I cannot give you a clear answer.

Earlier in the day, for example, I saw how I could get a (purported) "free ride" with an email "blast" to thousands of people, many of whom have asked to be removed from my mailing list.

The extra mailing would take just a few minutes to set up, and would not cost a cent financially -- at least in the immediate sense. And I probably would recover some valid names and maybe a few clients, while others, perhaps fuming at the spam, would simply hit "delete" or send a spam complaint to the mailing service, not me.

Why not go for it?


But I hesitated, and then rightfully decided to stop. Maybe a follow-up or verification mailing can be justified, but it needs to be handled carefully and the intent of people asking not to be bothered again by me must be respected. Until I can scrub the list clean of anyone who asked to be removed, the email won't be sent. And if I send it, I will think hard and fast about the content and relevance first.

Then, late last night, one of my talented representatives saw for the first time the "send" list of a regional construction association. We certainly don't have permission to use this list for advertising or marketing -- but he deduced that he could send out an email inviting readers (if they wish) to submit suggestions for a guest column or editorial contributions to the publication. It is unlikely anyone will be offended by that kind of offer. A trick? Maybe, but good marketing as well (as long as he doesn't routinely repeat the exercise with the same list.)

So, what, after all, are the rules here?

In the perfect world, I suppose, we would all only send and receive marketing messages we really wish to get. And when we achieve this perfect match, we almost inevitably succeed. Consider the natural flow of personal relationships, word of mouth, and inbound inquiries.

Potential (and current/former) clients want to do business with you, so they seek you out. Undoubtedly this is the easiest and most enjoyable way to sell anything.

However, most marketing communications efforts involve some waste and (worse) irritation of people and organizations who, often for truly valid reasons, will never do business with you.

This sort of thing happens when you "force your pitch" on someone at a networking event (rather than focusing entirely on the person you are speaking with's real interests), in intrusive advertising, telemarketing, and (painfully) spamming. Telemarketing and canvassing probably are the most intrusive approaches, but they come with a trade-off -- the need to personally interact with the person you are troubling. So you either pay handsomely for the canvassers or callers to "take the rejection" or you endure it, knowing that you can gracefully back off and move to the next "prospect".

Spam, frankly, is one of the most challenging (and perhaps tempting) options. If you have a list, it costs virtually nothing to force it out on people's email, and you don't (directly) have to face the rejection. So why not just do an email blast!

The cost, of course, is in your reputation and your chance at a future relationship. Remember as well, spam can be on a small scale. Some Public Relations agency, trying to interest me in a marginally relevant story, sent the same release -- and follow-up "personal" email to me and two employees of my organization on leave, so all three emails landed in my mailbox yesterday. I deleted them all.

But another PR representative sent a straightforward news release, without any purported personalization, to more than one address. Most of these announcements land in the round basket (okay) but this one stood out -- I followed up and will probably write a story on the topic. When does spam become newsworthy?

I believe the more you can engage people with good stuff and the less you resort to intrusion and disrespect, the more you will succeed at your marketing. But sometimes it doesn't hurt to break the rules. Just be aware of the risks and real costs when you do.

Testing your ideas (2)

Yesterday, I received a solid reminder of the costs and challenges of surveying and testing marketing ideas, especially when you engage in intrusive communications. The question, always, is whether the intrusion's reward is worthy of its cost.

The question is whether our special advertising-supported editorial feature concept, the backbone of our existing print business, could be adapted effectively to leads generated from this blog in the new Design and Construction Report (http://www.dcnreport.com).

You can see the original posting explaining the concept here.

Out of 1,419 survey invitations, 15 answered the question, but disturbingly, six people reported my email as "spam" -- the highest number of spam complaints I've received for a single mailing since I began using Constant Contact about two years ago. (The spam complaints may have resulted in part because I inadvertently selected one of my employees as "sending" the email, and the recipients may not have known her. (She got a surprise on waking up to find dozens of 'bounces' in her email box!)

However, while nine people responded by saying they would welcome publicity only without any cost at all to their organization (hardly of value for our business), three said "yes" to the question of whether they would be willing to pay for the service with a "satisfaction guaranteed or there is no cost" fee, and three said they don't know whether publicity would be of value to them.

Later in the day, Chase observed in an email to me:

"Three leads where people would pay a fee is promising.

"The real question is to determine if the people responding are the real decision makers or not.

"I suspect we would see an average of five per cent close rate from the blog leads. I have no problem working these and as more issues are published the close rate will increase."

The results, obviously biased with responses from people who have a relatively close connection with our business, suggest there is some interest in the concept, but we don't have overwhelming or obvious demand. In other words, I certainly have not hit a home run.

And, to get the answers, I pushed myself into the face of busy people who don't really need nor want the intrusion.

But it isn't a dead loss either. As Chase points out, if we can achieve a five per cent close rate in the early going (that is one in 20), without being a pest or spending too much sales energy on weak leads, we have a viable market -- because this blog and its high Google rankings generates upwards of 20 to 30 inquiries a week. With our average revenue per feature, this indicates a market potential of upwards of $100,000.

In marketing, sometimes we need to push out a little to see what is happening, and sometimes we need to be a little in the face of our current or potential clients. This is certainly not the "Permission Marketing" advocated by Seth Godin, but I think a totally passive approach with no assertion or invitation/call to action is asking for silence when you try out new things.

As well, I appreciate that our business idea probably requires the effective intervention and co-ordination of intelligent sales representatives. The editorial publicity/special advertising feature concept is reasonably simple, but may not lend itself to the "check the box, 'I'll take it'" attitude.

Nevertheless, I'll be respectful and careful before rushing to generate another survey. I need to respect that far too many people resented the intrusion.

Discretionary spending and marketing

Charles is my newest nagger. He's keeping a close eye on the company books, helping to impose some discipline in our operations. And lately he has been reminding me at every occasion that every dollar in discretionary spending requires $7.35 in revenue to offset.

In other words, he says if I buy a business book for $20.00, I should appreciate that we will need to spend $147 to pay for it.

"Go to the library," he said. (And I have, and picked some really interesting books I'm reading now that will cost exactly $0.00 in sales to read).

So what do you do about your marketing budget?

The advertising, new business cards, website redesign, your business lunches with important clients, golf games, and so on are all discretionary expenses. You can spend them or cut them, often at the drop of a hat. (Your rent, employee's salaries, and the like are much less discretionary -- sure you can change these costs, over time, but they are much more fixed than discretionary.)

Of course many discretionary expenses make real sense IF they will help you achieve the sales to the ratio you find is right for your own business. If our business spends $100 on advertising and achieves $735 in new sales, we're fine.

When you look through your discretionary expenses, you'll likely find real savings, however, on the things you don't really need, your "nice to have" luxuries, your lazy purchases because they seem small -- but aren't so small when they recur every month. And sometimes you find surprises.

For example, we've been spending about $200 a month on Constant Contact fees for our e-letter. Not much for a multi-thousand dollar business. I brought the fees down by $75.00 a month simply by removing invalid emails and some really old names, reducing the list to below 10,000.

Good, you might say, but could we do better?

I noticed that only about 20 per cent of the people receiving my emails are actually opening them. What would happen if we took measures to boil the list down to the number of readers who really want the emails. Now the list is less than 5,000 -- and the Constant Contact fees will drop even more.

But has our effectiveness diminished? Not one bit. We simply aren't spamming people who don't want the emails but haven't bothered to simply say "stop".

In watching your discretionary expenses, don't cut your marketing budget. Just make sure you are spending it where it will provide you with results and real value.

Are they really interested?

Just two weeks ago, our email list included more than 11,000 names, and seemed to be in a trajectory of unlimited growth. Today, the list has 8,500 names but soon upwards of 5,000 will be removed.

Is this marketing suicide?

Your database includes a list of 1,000 "potential clients" for construction, renovation or related services. The names come from "somewhere" but you really don't know them at all -- and they haven't responded or communicated with you in any (serious) way in the past couple of years.

Are they really potential clients?

Sometimes we accumulate names -- or on online services "friends" -- way too easily and superficially to build any kind of meaningful relationship. We diverge from our priorities, and waste money and time trying to convert people who have no real interest in our services.

I suppose if you are maintaining the names on some sort of free service and not spending any money keeping them on file, no harm is done. But would you not be better off marketing wise to have a smaller, but meaningful and manageable, list of real prospects, current and former clients, with whom you can develop rapport, relationships, and new business?

The new way for online construction marketing

We've just wrapped up (at 11:30 p.m. on Friday) an intensive online business session, where I and two of the company's employees, Chase in St. Catharines, ON and Karen Buckley in northern Virginia, engaged in some pretty intensive and resourceful email exchanges.

Of course, I don't expect this company's employees to be available for work near midnight as the weekend starts, but the Internet makes this sort of thing possible and feasible. Individually, we're just catching up on our business; with emails and web reviews; it just happens we're all working collaboratively as midnight approaches.

Why not?

These observations provide important clues for your marketing approach in 2009. The online space operates with astounding geographical and temporal range. Sitting in our family rooms with wireless laptops, we work in spare minutes off hours and get lots done.

You can use these tools -- and respect your clients -- by sending personalized emails, images, even estimates, any time day or night when it is convenient to you and your clients.

Note I am not suggesting you spam or send junk email; and if you wish to use an online newsletter or any other form of mass communication electronically, remember the importance of obtaining permission first.

But personalized one-on-one emails (where appropriate with individualized image or video links/attachments) are powerfully effective for marketing because they can reach your potential and current clients when they otherwise would be unreachable, without intruding in their space or disrupting them when they don't want to be bothered. This is much different than the irritation of having to go to the door to say 'no' to a canvasser or answer your phone from some ill-informed telemarketer.

Construction marketing: The success irony

Earlier this week, I emailed about 1,000 current and former print advertising clients with a simple offer -- they could receive an hours' free marketing consultation, with absolutely no obligation to purchase anything and with the understanding the consultation had nothing to do with selling our (print) advertising services. These clients receive the Construction Marketing Ideas newsletter and this blog.

Certainly, no one who received the email offer minded it -- no opt-outs from the mailing list, no cancellations, or the like. But only three accepted the offer, and each of the three is truly at the top of the game when it comes to marketing their services. They don't need my advice: They are doing things so well I cite them as examples of how to conduct their business.

Why is it that the people who could use marketing advice and guidance don't take it; while the people who really are doing well are interested in expanding their knowledge?

An inexpensive approach to online construction marketing

This posting from CF Construction on a Contractortalk.com forum thread Advertising for General Contractors is worthy of repeating here for its simple and elegant insights. I've only modified the grammar slightly (it isn't perfect), but if good writing is about clarity of message, these remarks are brilliant.

What we're doing, since we've decided to put off our magazine ads for perhaps the next quarter is simply e-mailing prospects. Since our demographic are individuals whom for the most part aren't being affected directly due to the recession (at least not struggling to pay the bills, or put food on their table).

I've purchased various magazines that lists countless of businesses down here in South Florida (i.e. luxury hotels, restaurants, spas, corporate offices, medical offices). I send a very brief e-mail "introducing" those prospects to our company. I invite them to visit our website so they can familiarize themselves with what we've done in the past and what we're capable of doing. Surprisingly, this approach has worked very well. Out of 10 e-mails, I perhaps average about 5-6 e-mails replying back to me. Which in return has resulted in a couple of jobs over the $100,000 tag to be completed within a month or so. Thus, giving us a very nice profit at the end.

If you have a website, you definitely want to make that very "search friendly". This site and some of its members have provided very useful tools to the point that we're getting many calls due to the website. In some instances, we're listed in the "local business" portion of Google searches, which are the ones way up top. It's just a matter of being dedicated, patient, and optimistic.

There are many other ways and approaches I've taken to ensure that we don't find ourselves waiting in the office for the next phone call to see if it's the one that finally gets us busy again, but this is pretty much the easiest one I've done so far and by far the cheapest. All you have to do is invest your time and a well written "introduction" and people will call.

CF is Claudio Fernandez, owner of CF Construction and Remodeling, Inc in Miami. His website is http://www.generalcontractorinmiami.com/