Mobile APM: Experiential vs. Operational

I mentioned before, mobile APM is different. One aspect it is different in, is the structure of the delivery chain, which commonly includes the wireless carrier.

The complete mobile delivery chain (Image Courtesy: Compuware)

When you think about the end user experience, unless their on wifi, performance over the wireless-cellular network is almost chaotic. Even optimal performance, when you have good reception and reasonable data protocol, you are still bound to the doppler effect (basically at any second signal reflections and cancellations can create temporary signal drop even if you're standing still in good coverage area). Compared to the ISP delivery model, cellular interfaces are creating a headache in trying to maintain mobile APM. According to Shunra, over 60%of application failures are due to non-functional issues. Sometimes you want a "clean room" environment.

"Clean room" environment allows quick isolation and correction of the APM issue (Image courtesy: Compuware)

You still need to maintain the monitoring methodology from the end user perspective. Why? because all components making the experience come together in the browser/app. In mobile, typically there are more 3rd party vendors (which you need to watch!) compared to desktop.
So you ask yourself, should I include the wireless carrier in my monitoring, or monitor via a connected ethernet line? Here is my recommendation.

• When you're trying to identify, isolate and address issues, you want to perform operational monitoring over the ethernet (eliminate noise from the cellular link). Why?
• Most issues have to do with backend issues which you can correct, and those manifest themselves in the same way on wired connections (you can think of a wired connection as the equivalent of iPad on wifi).
• Issues related to cellular link performance are highly intermittent and you can't fix them (unless you're the carrier or someone the size of Google, and even then.).
• When you're trying to optimize the experience, understand the true end user experience, perform experiential monitoring (using real user experience data that includes the cellular and wifi links). Why?
• Because you can understand the end user experience globally and tune your CDN, composition of components on your page/app
• You can better identify trends and address those by dynamic tuning of the backend

Naturally, experiential monitoring is best served by tagging apps and websites ('Real user monitoring') like many companies offer (Compuware, Google, Localytics, Flurry, Yottaa etc.).

Hope this helps, interested in your thoughts!

A Winning Product Manager: The Organized PM

Image Courtesy: Go Planet Go

One of the things I stressed to PMs I worked with is the importance of being on top of your product, having the verbal and written supporting knowledge and materials. You have many constituents (internally and externally), most don't report to you. Your job is to acquire their confidence in you, and maintain it. Being organized and knowledgeable helps a lot.
It also helps you. If you're nearly as successful as you want to be, there will be immense pressure from all constituents to consume your documents. You're helping yourself if you can find them quickly.
Therefore, my recommendation is to keep an organized folder of updated documents that describe your product in details:

• Business aspects
• Complete P&L excel of your product: know all of your product expenses (CAPEX, OPEX), know the pricing structure of your product (sold independent and as part of the portfolio), and your margins in different deal levels. Know your current expenses and revenues.
• Business Plan: Know the sales forecast of your product, by sales team, region, deal size etc. Base your assumption on current user base. 
• Top customer and lead analysis: know your top customers perfectly. Document monthly calls/meetings with them. Document their top 10 requests and describe their roadmap and your translation into what it means to your product. Know their contract, and usage of the product. Note the top prospects for your product. Who is chasing them, what's stopping them from signing, what product and/or business gaps exist.
• Competitive analysis: note your current and future competition. Where you stand compared to them, what areas reflect most on your customers and prospects. Note investment areas by VC and analyst coverage.
• Product
• Product requirements and/or backlog: without going too much into SCRUM methodology, I believe every product manager should know by heart the required features for the next 12 months ('H1') and his customer needs on a quarterly level. If you're doing waterfall, 2 releases should be covered at 90% confidence. If you're doing waterfall, the top requirements on the backlog should be covered at the same level. Here, clarity is gold.
• Roadmap: Describe H1, H2, H3. H1 should be, as said, 90% confidence. You know you're getting it. H2 (12-36 months from now) at 75% and H3 (goes up to about 4 years away) at 50%. Strategic insight from analysts, industry and your vision are key here. Give execs a sense you know what you're doing not only at a tactical level but also strategically. 
• Key support cases: Many times PMs forget support (the 'ugly' side of customer interaction). In my view the number of support cases relates directly to your ability to control churn and gain more traction. The list of support cases is a key driver to immediate fixes or highest-priority requirements in next release/cycle.
• Marketing collateral
• Product brief, Portfolio brief, on a technical, value and business level: PPTs, one-page brief and the likes. It's not an overkill to have a version of PPT that can be delivered verbally, and one that's designed to be read offline. Ensure you cover business benefits and how your product integrates in the portfolio to drive a holistic solution, which is better than the competition.
• Customer use cases & references: Create compelling & educational evidence of how your customers gain value, increase productivity and RoI using your product.
• Some thought-leadership topics: if you were invited to a conference today, what would you talk about? What can you say that is A- thought provoking, B- you're pretty much the only one who could back it up with data and C- that is interesting and valuable to the outside world. Have 2-3 topics like that in your back pocket.

A product manager who has that collateral in their folder, is someone who, in my view, is on top of their product. Needless to say, backup and version control are key. You will have many versions of each of these ;)

What do you think? what else should be part of the product manager folder?

Explain that to me? (or: I like to poke at things!)

Preamble: We recently relocated to Israel. After 10 years in Boston.Big change. More later, perhaps, when the dust had settled.
One of the first things we did is connect ourselves to the internet. I know, Web = Oxygen for some people (me). I know what you think. First on the iPhones, then home.

My darling, dead Linksys router

Of course, one of the things I hauled with me was an excellent Linksys router. I plugged it in, then power. I noticed it's supply took it 110v, so I attached a converter. In the plug, a short hiss, leds turned on for a fraction, turned off, and the horribly, hated smell of fried electronics. Hip hip hurray. (Office Depot win 100 shekel: Amir 0).


Which brings me to the point (finally eh?): In 2012, why 110v vs. 220v? Increased market size for device diversity? creating more jobs? What's the point?
It seems like some things in the world are like they are and will stay like that simply by avoid cost of what makes sense.
There are no additional OEMs because there are different standards, the same OEMs install different power units. There are no customer facing features or benefits to the customer because of that. On the contrary: If you move between regions, as far as electrical devices go, you are forced to buy the device that will comply to your target region pretty much. And you will pay more for the same value.

I want to poke at that. I want someone to explain the logic to me.

Here's another one. In Israel, many consumer products and services are small, not the best quality. Why? IMO, it has to do with buying power. It's the cost of importing cars, computers etc. into a 5-million people island (I'm using the word 'island' because economically, Israel is on an island. No economical cooperation with neighboring countries.). Toyota can't bring here & sell (in a reasonable price) their Camrys and jeeps because it's too expensive for them, because we look nothing like EU, Canada or the US. No buying power. Ok, Israel can't build economical relations with Syria and Lebanon (and I'm not interested in any political discussion), but what about Egypt and Jordan? What if all 3 became a multiple of consumer power? What would strong economical relations do to political relations? Economical relations would strengthen the political ones. So I say, fix this, for the benefit of everyone in the region, the OEMs and service providers. Let the region flourish.

Here's a bigger topic: driving on the left or on the right. In past jobs I was fortunate to travel. Yes I'm an addict. Sometimes I was on a red eye to London, and I had to rent a car. Imagine flying from a left-handed steering into a right-handed steering on a red eye. My solution was to always drive behind someone. But I was on a 747, I was not alone. I don't know the numbers, but I argue there is human price for this idiotic fact.

And don't tell me it can't be done. Man landed on the moon, Google are inventing self-driving cars. It can be done, and it's the right thing.

Side note: if you work with me, you'll find that I rarely 'accept' challenges at face value without really understanding them. I'm passionate about driving differentiated value to the customer, so spend the time and convince me the existing approach is the right one. Otherwise, help me change it ;)

A Complete Mobile QA/Application Performance Management (APM) Strategy

Image Courtesy: Flickr

You want to create and maintain a valuable, optimized mobile service to your audience. There are different solutions and approaches for effective performance assurance of your mobile service over time. It is important to understand and optimize the performance of your mobile service in different scenarios: pre production, post production, under load etc. A more modern aspect of mobile data is the rapid evolution of rich media content (not only in the context of entertainment, but also as brand promotion, product description and reviews etc.).

This post describes different mobile APM and QA solutions and what constitutes and complete solution. To start, I'd like to describe several macro-trends and challenges that affect the space.

First, Mobile is unique ;).

Retailers/marketers ability to offer targeted services to their customers on mobile is better than desktop. That highly targetable audience had high expectations: in 2011, Compuware found that 71% of all users expect mobile services to match or exceed the availability and performance of desktop experience. Since the competition is around the corner, failure is expensive. More expensive on mobile.

Compared to services delivered to desktop users, mobile devices are highly fragmented: screen sizes, OS variants, OEMs and device features, behavior under different 'environmental' conditions such as the wireless (cellular/wifi) signal and protocol, number and composition of applications running on the device, driving resource consumption (memory, battery, throughput) etc. Further, mobile phones and tablets are not production-testing designed devices: under ongoing testing, they will suffer inherent stability issues. These certainly aren't enterprise-quality servers. To add, phones and tablets are expensive! Scaling a QA/APM solution is challenging.

Another uniqueness of mobile, specifically in B2C scenarios, are resident apps. Most desktop experiences are delivered via "standard", limited set of (almost) browsers. Apps are more difficult to QA/APM. A former carrier employee once told me: "The trick is not to test the phone or app. It's to test it in random combination with other apps".

Another interesting trend is recent closer integration between development and operations. Deployment cycles are getting shorter and even 'heavy-enterprise' applications are more open for experimentation in return for user feedback as genuine contributor to the optimal product composition. As a result, functional testing and application performance management are becoming closer. Last, true rendering of mobile experiences are an integral part of the QA cycle.

The market wants a unified solution: QA+APM. The breadth of the deployed solution depends on a growing maturity of the Dev/IT/Ops organizations in leveraging value from the solution. At all times, RoI from functional QA/APM solutions needs to be clearly demonstrated to the LoB (prevention of outages, revenue/customer loss and brand damage).
There will likely be more than one technology buyer. A QA/Dev representitive will usually make the initial purchase. Initial APM buyer will be somewhere between QA and IT/Ops. Eventually, every mobile APM strategy will become part of the overall APM strategy and owned by IT/Ops. Assuming functional QA starts with physical phones, the interest will be to proceed with real-device based APM.

In my view, the ultimate mobile APM/QA solution consists of the following toolsets, integrated into a central reporting solution:

• Functional Mobile QA & Automated Testing: Pre Production testing of mobile services, on real handsets. These can be offered onsite or remote (dedicated or shared). The main point is to run tests on real phones in as many scenarios as possible, representing use cases, environmental parameters (location, carrier etc.). I call it "Clean room QA and regression". While testing on real devices, ensure you get screenshots or even videos of the transaction. Companies who offer this service include Perfecto Mobile, Jamo, Device Anywhere (Keynote) etc.
• Crowd Sourced QA: Unlike the structured QA, crowd source QA can offer insight to issues caused by human users in unplanned scenarios, which may contain use cases, combinations of applications running on the phone, different locations, carriers etc. Companies who offer this service include uTest. Unsure who else ;)
• Proactive APM: Ongoing monitoring of mobile services, from the end user perspective. Both pre and post production. These are robots located where the user is, running recurring scripts. There are several options: Real phones/tablets vs. simulators. Onsite vs. SaaS. Ethernet vs. over the cellular carrier. My preference would be toward simulator-based remote monitoring in a SaaS environment, using both the wireless carrier and a wired connection. Very close to the end user experience. Simulators can be instrumented to provide signal strength, CPU consumption, application metrics and much more. It's cost-effective with lower management overhead. It's a clean room environment for ongoing monitoring. Who provides this?? unsure.
• Real-User based APM - SaaS: This approach lets your users work for you.  Passive tags are instrumented by the service owner into the app/website, and as your users access those, data is generated, describing the end user experience (therefore, it's all post production). This approach is scalable, insightful and real. Moreover, W3C is defining timing APIs for browsers as well as resource timing APIs. This is really exciting: imagine mobile users could submit complete HAR files describing the real user experience. There are a lot of startups in this space because all you need is simple (relatively) collection tags, and the heavy smarts is in the data analysis. It's been a pleasure hearing a retailer saying: "For as long as I was proactively monitoring my site, I had no idea of the countries, devices and experiences my users were going through. As soon as I turned on real user monitoring, a whole new reality dawned on me. My users are mobile, they travel and consume my service from everywhere, specifically from places my proactive monitoring does not exist". The SaaS approach here really means that there are no on-premise installed components. So you have limited insight on what happens on the backend. Companies offering this service: Compuware, Yottaa (I think), others  
• Real User-based APM - On-Prem: Like the above, except there are on-prem installed agents or appliances. The big difference is the ability to correlate between the transactions users generate and the response time of each server in the backend. It could be extremely insightful and help identify and resolve issues quickly. (It's worth though keeping in mind that a lot of mobile services leverage 3rd party components. So if your 3rd party vendor component does not perform, these on-prem agents you have installed on your backend, really don't offer much insight.). Companies offering this service: Compuware, others.

In the spirit of optimizing an app/mobile website, you can't forget analytics. So important. What's really exciting (to me) is advances in analytics done by W3C (timing APIs) and really cool innovation like what ClickTale are doing (seen it in action, it's awesome!)


OK, this came out a bit long, but I hope it gave some insight to whoever was looking for APM+QA solutions for mobile. Nobody has the complete solution, I'm sure VCs are creating complete portfolios around similar concepts.

As always, very interested in your feedback!

(Mobile) Topics that get me excited these days

You may have picked up, following me relo to Israel, I'm on the job hunt these days. Relocating is a full time job. But, on occasion, I get a free hour to connect to the web, and I come across exciting spaces and solutions. Here are some that excite me (AKA, I want to product manage ;)):

• App store analysis and discovery: Social sharing, recommendations and discovery of apps is just not where it should, or can be. Some startups are taking on that challenge: Appsfire, Quixey, XYLogic and Chomp (now Apple).  More here 
• Real World eCommerce Link: Between (fail?) Google Goggles (which I had huge hopes for) and QR/other barcodes (really? show me how to monetize) nobody has been massively successful in building a complete an eCommerce solution that recognizes physical objects and offers shopping experience around those. I've no doubt it can be done, it's just a solution that consists of many complex pieces. I've seen interesting technology by SuperFish. I think this space has huge growth and reward possibilities. 
• Advanced mobile analytics (dare I say combined with APM and production quality?): In mobile I believe there is huge opportunity for advanced analytics. Localytics, Google and others are doing a great job at it, but there are two areas (IMO) for improvement: 
• Mobile usage 'heat map': Understand what your users are really looking at, what exactly works and what doesn't, and what's the impact on your success criteria. One company that does an interesting job: ClickTale. I've seen them in action in NL Facebook mobile equivalent. Massively impressive. Again here, I believe there's huge areas for improvement in market awareness.
• Integration with APM and production quality solutions: Yes, most likely there may be multiple audiences. Marketing will look at straight analytics and understand how it impacts the success criteria. IT/Ops will look at the latter. In reality, your bottom line is impacted by both and this is understood by the customers (who are approached by all players in these spaces). So whether you're an analytics play (Localytics, Google), APM (Compuware, Yottaa), Production quality (Crashlytics, Crittercism), you really want to be thinking about integrated services/dashboards/APIs etc. 
• Enterprise solutions/BYOD/Virtualization
• Enterprise solutions: I personally believe the space for B2C games/music/productivity is limited. There's just so much there already and the ability to make it big is just slim. An easier way (to innovate at least) is to find what apps Enterprises need and offer that to them. Easier said than done, but the potential is there. I've been inspired, for example, how General Motors, Lowes and Home Depot, Verizon and others leverage mobile as a sales associate tool. Is there still opportunity to innovate in sales force enablement? Absolutely.
• BYOD: I think there's some debate around the BYOD opportunity. Like coffee, people are split into extremes on this one. Clearly, it offers an opportunity to the enterprise (and a headache to the CSO). Enablement of BYOD in the organization requires a complete MDM solution, which is very complex (remember how touch it was to deploy Blackberry?).
• Virtualization: I can't tell you how ridiculous it is to see folks (I've seen them mainly in EMEA) walking around with 2 or more phones. Good for the operators, I suppose, but not so great for the person and an opportunity for their employer. Virtualization enables one phone with multiple personalities (+high walls between them). Whether BYOD or employee-provided phone, it's easier on the employee, and the employer can potentially gain some of the employee attention on the off hours.
• Social Navigation: I'm a bit split about this one. I'm a constant user of Waze. What they do and the value they provide is unparalleled. The only problem with the solution is that it depends on user input. The input needs to be entered on location and in real time. OK, you may be in a traffic jam and there's no big damage if you look at the phone (illegal!!). But if you're driving and there's a hazard, that's downright dangerous. I think that if input could be handled verbally (A-La Siri?) that would make it all very attractive, and safe.
• Personal interest: Way out of my field of expertise, but there is highly interesting 3D printing technology being created. Fascinating, and feels like rocket science. One I took a look at was EZ3D. 

Of course, this is a partial list. Interested in your thoughts, what do you think of those spaces? What excites you?

Top 10 Characteristics of a Winning Product Manager: Preamble & 1st Post: "Product Manager: The Eternal Optimist"

What?

If you're lucky (like me) to travel between Europe and America, you know the world map is presented differently. Each region draws itself as the center of the world.
I think product managers are the center of the organization. Good ones lead winning products and inspire winning teams.
I've decided to spend some time summarizing what, IMO, makes a strong product manager. While my expectations from a good product manager are well defined (in my head), what characteristics identify a good one (or how to identify your next PM recruit) is more challenging. Seeing a few PMs in action, I thought I'd put some of my impressions down.

Why?

Really? another blog post on strong PM skills? YAWN.
Here's why. In a ocean of free digital media, wealth of information, blog contents rarely are original. What's original is the author's personal view. Here are a couple example of blogs I love: Fred Wilson, Bijan Sabet (+his Hallway chats with Nabeel Hyatt), and others. Recent jewels are Hadas Sheinfeld and Guy Nirpaz. Their blogs are worth adding to your reader because they are educational (technology and business) and personal at the same time. I'd like to think I can be interesting like that too.

I split the "Winning Product Manager Characteristics" post because you won't read all of this. Posts need to be short and to the point.

So, w/o further ado:

A Winning Product Manager: The Eternal Optimist  

Image Courtesy: People-Results.com

A succeful product manager is a team leader. They lead a team internally (everyone who builds the product and sells it: Eng, QA, Sales, Marketing etc...) and externally (customers, analysts, partners and channels etc.). But they are not a formal team leader: nobody reports to them. They lead by vision, optimism and ability to inspire. The product manager shares the successes and gaps of the product in the most transparent manner, and drives enthusiasm around the ability to drive that unique value.
Therefore, a product manager has to be the ultimate optimist. Their belief in the success of the product has to be fundemental and eternal so they maintain their optimism and spread it. Optimism is contagious (and therefore can be precieved as a tool).

Is this the most important feature of a strong PM? Questionable. I chose to start with this one because when you're hydrated, in good spirits, your brain will work out the toughest of challenges. Back to basics. Notice how inspiring leaders are always positive.

Personally, I'm not the eternal optimist, I wish that was different. So I use and recommend adopting a quick dictionary, and interpolating it. Here are a few samples (in no particular order):

• The competition is doing better = The market size is growing. The competitors are showing you one way to be successful. You have an opportunity to play this game. Figure out what they did well and improve on that.
• CxO wants a product review = Executives are interested in your product and want to help. Leverage this opportunity to harness them into what you need most. Don't tell them pretty stories, give them the truth and get them to work for you.
• I'm not getting CxO attention = They trust you and your judgement. You've been given resources, right? go make the best of it, show results and make sure to make that visible such that customers demand increase in product features and investment, and be willing to pay for it.
• I'm bogged down by thousand of sales calls = The sales force is fully engaged. They (think they) can make 'Club' selling your product. More importantly, the market is thirsty for your product. Teach them how to fish: participate in a few calls, raise the knowledge and more importantly the level of confidence sales reps can have in your product, and let them sail away. Early time investment with your sales force is a well worth investment that will pay off big time. Travel with them, be their friend and let them buy you a steak ;)
• Engineering are hard-headed = Engineering are smart, passionate and are eager to contribute to the overall success of the product. Leverage it in the product requirements, go to market, whatever. Reach technological understanding with them that makes them feel their opinion is heard and considered, and that they are important to the overall success. Make sure the areas of responsibility are kept clear (you answer 'WHAT' and they answer 'HOW'. Moi importanta. more in a separate post). You'll learn much from engineers experience, knowledge and insight and your product will be better.
• The product has huge gaps/bugs (usually, vs the competition. Sometimes against minimal shippable product criteria) = Your product potential is huge. And, you know what's missing. Prioritize it, break it down, define your roadmap. Show the case to everyone around you: here's what you get for this investment. Make as much advancements with what you have and incremental, customer-facing meaningful deliverables. Show the world (and yourself) the potential is really there.

OK, 6 little common 'translations' I've adopted to keep my chin up, and to motivate others.

What do you think? makes sense? How do you keep your team (=internal + external, and I mean customers and analysts as well!) motivated and inspired? How important you think subjective optimism is? How do you keep optimistic at tough times?

A penny for your thoughts!