Blind Data Dilemma

By Hank Berkowitz • March 9, 2015

Hank Berkowitz, Association Adviser
Hank Berkowitz, Association Adviser

With Valentine’s Day in the rear view mirror (sadly for some, gratefully for many), relationships are on many people’s minds. We’re human beings, and we have complex relationships with friends, family, colleagues, significant others and even our rivals. We also have complex relationships with our members. They’re miffed if you forget about important dates and milestones in their lives. They’re annoyed if you don’t reach out often enough to show how much you care. At the same time, they get turned off quickly if you overdo it and don’t give them enough space. It’s a tricky balance that’s probably more art than science.



Most members will share their experiences with you if you ask the right way and provide access to the results (preferably quickly). LilTweetablesSmall

What members tell you in surveys and how they actually behave in real life can be two different things. You need qualitative input, not just quantitative. LilTweetablesSmall

Associations often overlook the engagement a member has with an idea—it’s the mental and emotional connection they have with the core notions of what your association is. LilTweetablesSmall

As many a membership director has lamented over the years, “members complain that we send them too much information. At the same time, they complain they never knew about a certain event we were hosting or a certain benefit they were entitled to.”

Sound familiar?

Information overload and communication clutter have been the top association challenges every year since we started our annual communication benchmarking study in 2011. But simply turning off the communication spigot isn’t necessarily going to help you. You have to be more targeted and more efficient. More than half (55 percent) of the 1,000-plus association execs who participated in our 2014 study told us they have to explain member benefits more effectively, and one-third (34 percent) need to keep members better informed about the events and professional education opportunities they offer. What’s more, nearly half of respondents (48 percent) told us they need to do a better job of helping members find information quickly, and a similar cadre (46 percent) know they have to do a better job of customizing their communication efforts to different subgroups.

Top Member Challenges - Benchmarking 2014

So what’s the answer?

At a recent Association Adviser webinar I participated in, my fellow panelist Jill Andreu explained that simply communicating less often isn’t a magic silver bullet. You need to assess, integrate and measure what you’re doing. In other words, improve your “AIM.” As our benchmarking research confirmed, only one in 10 association leaders (11.5 percent) felt “communicating less frequently” was a key to better member engagement. Andreu, who is Naylor’s VP of content strategy and development, said more effective tactics include getting a better handle on the specific types of content expected by members (62 percent agreed), integrating your messaging across channels (40.3 percent agreed), enhancing your data collection/usage capabilities (36.4 percent), improving social/mobile engagement (27.3 percent) and knowing more about members (21.8 percent).

Benchmarking graph Q18


Notice how many of those tactics above are related to data (enhancing collection/usage capabilities and knowing more about our members)? That brings the whole notion of “engagement” into question. Everyone from Fortune 500 consumer brands to single staff associations is using the term “engagement,” but it sure is hard to pin down.

Steve Rappaport, author of the new Digital Metrics Field Guide, told me last week that the word “engagement” tends to get thrown around a lot, and it typically means “actions that people take to interact with something, such as clicking on a link, completing and submitting a form, commenting on a blog post, and so on.”

However, Rappaport said you also have to pay attention to the engagement that a member has with an idea. “It’s the mental and emotional connection a person has with the core notions of what an association is,” he said. The strength of that engagement can influence all sorts of decisions, from joining, staying or leaving to advocacy, participation, etc., added Rappaport, former knowledge solutions director for the Advertising Research Foundation and an adviser to corporate and not-for-profit digital marketers.

Big data isn’t necessarily better data

So, before you rush out to hire a data scientist or invest in the latest statistics or predictive analysis packages, keep the big picture in mind. That’s why the Ohio Society of Association Executives only looks at three-year trends when analyzing their conference attendance rather than year-to-year comparisons. As OSAE head Shane Yates explained, you can’t just say that this year’s venue or program content or entertainment was better or worse than last year’s. You also have to take into account other schedule conflicts and events (both from inside and outside your industry) that could be affecting members’ decision to attend or not.

See our Corner Office profile for more insights from Yates. 

And before you collect a single data point from your next survey or research initiative, remember what’s in it for the respondent to share with you. They will participate in both traditional and non-traditional research gathering initiatives if they know they will have access to the data and results you collect—especially if they know they’ll get results quickly.

Susan Cain, co-founder of Quiet Revolution, LLC, explained in an ASAE keynote address that many members, especially the introverts, keep careful mental track of who the givers are in their lives and who the takers are. Maybe that’s why one of our most popular stories over the past year was How Can You Show Association Members More Love? As author Kelly Donovan Clark explained, “Most members will be happy to share their experiences with you if you ask the right way. Also, high performing associations use a combination of quantitative and qualitative techniques to understand their members and connect with them more effectively.”

Andrew Berkin, Ph.D., director of research at Bridgeway Capital Management, who’s active in both the national and Los Angeles Chapters of the CFA Institute, spends his day steeped in numbers. But he agreed that good data-driven decisions come from a blend of qualitative and quantitative activities.

“Evidence-based research means backing up conceptual arguments with the appropriate data to arrive at a decision,” he said. “In other words, When the story and the numbers come together, that’s when you gain the most conviction.”

Berkin’s new book, co-authored by Larry Swedroe, is intended for investors, but it has merit for anyone working in a highly competitive marketplace. Don’t waste time trying to outsmart the crowd, the authors advise, as the crowd most likely has all the information and tools that you do. Instead, use data intelligently to find isolated battles that you can win—that includes associations that are seeking to recruit or retain new members, especially up and coming members of your industry or profession.

When it comes to millennials, Rappaport said associations should adjust their marketing for each of three segments:

  1. The person who sees the association as a partner in their professional growth and career;
  2. The person who is an information hound and values the organization for access to knowledge;
  3. The person who sees the organizational membership as a status symbol.

As illustrated in the chart below, our research shows that association leaders are finding some of the most effective ways to get member feedback come more qualitative than quantitative methods, such as face-to-face interaction at conference, town halls, live events and annual meetings, and by mining your member service call logs, your website comments and social media forums.

Benchmarking graph Q26

Mind the gap and your endometrics

A communication gap analysis —a report that documents how closely the goals of an association’s communications plan align with the needs of members, vendors, advertisers and sponsors—can help association staffs uncover gaps between perceived and real strengths and weaknesses of their communications programs, writes Clark in today’s issue. Knowing these gaps allow associations to plan appropriately for future content and the advertising opportunities and roles that can accompany content platforms.

“A gap analysis should be the first step in any non-dues revenue program. Know yourself, and know your audience,” observed Tara Ericson, Naylor group vice president.

If you’re monetizing your communications channels, also make sure you’re mindful of the “endometrics” trap. According to Rappaport, endometrics is a fancy word for measures that come from within the system that’s being measured and report its performance.

“Pins, Likes and reach are just three examples,” he said. “Unlike scientific measures that come from tested and accepted theory, endometrics come from ideas about how a platform works, the features and functions available to users, and the business model for selling advertising.” As Rappaport related, “When a platform tells you that getting fans and followers is what you must do, most advertisers then chase them. But is that best for the advertiser? They should be optimizing their strategy to the platforms they’re using, and then selecting metrics that will measure its effectiveness. That can only be done when the association’s communications team has a clear understanding of how a digital channel and platform is expected to work for the advertiser or sponsor’s benefit.”

It’s all about the presentation

Finally after all that work you’ve put in to collect, crunch, analyze and synthesize your data, it often won’t become actionable unless you can present it effectively. As Naylor Marketing Supervisor Kent Agramonte explains in today’s issue, an effective way to present your results is through infographics.

Infographics are a visually pleasing way to condense multiple pieces of factual information into a single snapshot. What’s more, the information can be emailed or tweeted out to members or potential advertisers to let them know all the great information you have found and then later published in magazines that cover your industry.

Scott Fasser, a digital marketing guru for Hacker Agency, said that figuring out exactly the right metrics to present to directors, VPs and C-Suite execs is a special skill. “How the data is presented, how often it gets updated, what charts/graphs/visualizations are used and especially what story the data is telling is extremely important for communicating progress.”


Whether it’s in personal relationships, business relationships or membership relationships, at the end of the day you get what you give. That’s something associations can celebrate with members 365 days a year.

Hank Berkowitz is the Moderator in Chief of Association Adviser eNews.