If your IT shop always reacts to application issues, you're not making the business look good. Companies with proactive approaches to application performance management enable business wins.
Beauty product supplier ULTA Beauty overhauled its application performance metrics and management toolset as part of an architectural revamp for scalability and reliability on its e-commerce site.
The team styled an infrastructure-wide monitoring scheme that inspects code, monitors app performance, future-proofs the architecture and catches potential problems in development well before they move into production.
A head-to-toe e-commerce makeover
"Before application performance monitoring, we were very reactive and didn't really have many tools," said Michelle Pacynski, senior director of IT for e-commerce and marketing systems at ULTA, headquartered in Bolingbrook, Illinois.
Custom-coded alerts forced administrators to manually check that all pricing from the advertising system transferred, all the inventory was refreshed, data was where it belonged and so on. ULTA used ManageEngine application management software, but improper configuration meant that alerts "told you a little but not the useful information that you needed," Pacynski said.
They also used a console in their e-commerce hosting application Oracle ATG to monitor Java by thread counts. Once a problem was identified, a team member logged into their data center hosting provider's portal and evaluated system resources.
"We'd do a ping to validate that server boxes were up," Pacynski said. "If they were down, we'd go to the portal and look at each server one at a time: memory, CPU, file systems, etc."
To start application performance monitoring (APM) and management, ULTA established benchmarks with Compuware Gomez, a website performance testing tool. It hit not just ULTA's homepage, but also looked at end-to-end transactional and browser performance.
ULTA replaced ManageEngine with Compuware's dynaTrace APM tool on team members' recommendations, and ran a proof of concept on an application with memory management issues.
"That exercise got the team experience and buy-in with the tool before deploying it to ULTA.com," Pacynski said.
As ULTA's architectural overhaul took shape, they installed dynaTrace agents across the production, stress and nonproduction environments for full analysis.
At first, they only put dynaTrace on one of half a dozen app servers, each with three Java virtual machines (JVMs), but quickly realized comprehensive APM was the way to go.
"We only purchased enough agents to cover a portion of the environment," Pacynski said. "Very quickly we realized it wasn't going to give us an accurate picture," as attempting to use APM sparingly saved money, but lost the real return on investment.
The old process of sniffing out problems server by server in the hosting provider's portal is gone.
"CPU, memory and file systems information is more robust in dynaTrace," Pacynski said. "Because we have the agents in the JVMs, we can tie info from the operating system and middleware to the app tier and see where it relates to the code," she added.
If the problem lies with a particular functional aspect of the Website, ULTA uses its APM tool to optimize it, Pacynski said.
APM also proved useful for automated reverse-engineering, scouring the code that had accumulated from many different IT revisions.
The tool isn't for the novice IT admin, however. "DynaTrace is a very low-level tool; you're going to need some pretty experienced Java developers [to inspect thread dumps]," Pacnyski said.
She recommended investing in staff that wants to "find the needle in the Java haystack," and bring in the APM vendor's guardian services to deploy, refine and get used to the tool.
With Compuware's guidance, ULTA went through many iterations of its dashboards to reach settings that informed the right people of the right metrics in the right time span.
Work to be done in APM tools
Most companies only track app performance metrics on their top four to five applications -- SAP, for example -- as opposed to on all 30 or more enterprise applications in the data center, said John Enck, a researcher with Gartner Inc.
Tools from vendors such as Compuware, Riverbed, New Relic and AppDynamics are popular and successful, he said, and are replacing business service management products.
While APM has become more sophisticated, it hasn't maxed out on capabilities.
"I want some basic OS-level stats in the dashboard," Pacynski said, because tuning the OS improves application performance.
Future updates to dynaTrace will add a view into the OS and host level for viewing processes and system resources. Compuware is also working on automatically mapping infrastructure, catching common performance anomalies and pointing APM users to these problems.
"It's not an easy tool to use; people really have to spend time learning how to do drill-downs and how to find what they're looking for," Pacnyski said. "APM is a complex topic. Generally, I'd like to see APM tools do a better job of identifying or sewing together the story."
Still, APM provides a common language for people working together on an implementation, Pacnyski added.
Enck recommended using tools with strong end-user experience monitoring metrics.
"We're really good at watching stuff move around the data center and not so good at looking at where everything's actually happening at the endpoint," he said.