When the Apache Log4j vulnerability became known at the beginning of December, it was immediately all hands-on deck at BRYXX. Never before have we received so many messages from concerned customers. By automating our approach with Puppet Bolt, we were able to save time and provide our customers not only with the necessary information but also with a concrete solution in record time.
Vulnerabilities come to light every day, but it’s been a long time since one made the headlines. In early December, we were introduced to CVE-2021-44228: a zero-day exploit of a vulnerability on the Apache Log4j 2 Java library, which received the highest risk score. The danger was great and immediate because it turned out to be very easy to exploit the vulnerability. In addition, Apache Log4j is a very popular Java library, used as a logging framework for Java in many commercial products.
As soon as the news reached us, BRYXX held a crisis meeting. This led to a communications and action plan. First we sent an email to customers for whom we perform managed services. We explained the incident, not just in general terms but also specifically as it related to our customers who use Oracle and Talend products that contain Log4j. We also provided an overview of workarounds and links to sites where customers could monitor the status of the impact.
Automated approach
At the same time, we set to work ourselves to map the possible impact and come up with effective solutions for our customers. We are fully committed to automation through the development of scripts for Puppet Bolt. Initially we developed a script to list the customers using the affected versions 2.0-beta 9 to 2.14.1 of Apache Log4j. We were able to immediately reassure customers who didn’t appear on the list. For those who were on the list, we wrote a script to automatically implement the correct workaround. Finally, we wrote a third script to find out whether the impacted environments had been effectively exploited.
Thanks to those scripts, we were able to perform the necessary checks and proactively install patches on a large scale, simultaneously for all our customers. Our next step will be to share the scripts on Puppet Forge. This gives Puppet Remediate users access to the solutions we have developed at BRYXX. If the problem resurfaces in the future, those users can include our solutions in their automated vulnerability management — just like we do for our customers. We believe it’s important to share our expertise and contribute to a more secure IT world.