DevSecOps Series - Introduction

Nov. 26, 2024 // echel0n

Hello guys! It has been a while. I have been busy with other hobbies this year, focusing more on producing music. If you are also into electronic music, check out my playlists.There is certainly something lovable for you since the channel has a wide variety of genres! Hope you enjoy!

-1 Glossary

DevSecOps: (Dev)elopment, (Sec)urity, (Op)erations 
(S)-SDLC: (Secure) Software Development Life Cycle 
SCA: Software Composition Analysis 
CI/CD: Continuous Integration / Continuous Deployment
SAST: Static Application Security Testing 
DAST: Dynamic Application Security Testing 
Secret Scanning: Finding all those hardcoded clear-text passwords in the code-base or repository.
Feature-driven development (FDD): Model-driven short-iteration process that consists of basic activities
VM: Vulnerability Management, continous, automated process that is trying to keep your digital environment safe.
Boundary value analysis: Testing functionalities' inputs and their behaviour against valid and invalid partitions
Equivalence Partitioning: Selecting representative values from different equivalence classes to reduce testing effort

In this series, you will learn the bare minimum what you need to know about DevSecOps—at least you will be able to talk and discuss it confidently. Plus, in the upcoming blog, you will be able to set up your own DevSecOps lab environment! I will provide k8s configs for deploying to your server, including installation steps.

0 - Prologue

Over the last decade, everyone was hunting for penetration testers. Most organizations did not even have internal teams to manage these processes. Over time, these tasks expanded and matured due to local and global audit requirements. In other words, they became standardized.

Not that penetration tests are obsolete now, but organizations and individuals have learned a lot from them—especially about how ridiculously costly they are in terms of time and money. Even though we, the people conducting these tests, had a blast finding and exploiting vulnerabilities, the project owners and developers hated the costs. Then, they calmly asked the million-dollar question (in unison, I guess?):

Just kidding. They said nothing. Jokes aside, I tried finding the origins of this term, and I came across Shannon Lietz's blog, "Principles of DevSecOps". She covers the mindset and culture of DevSecOps and why it matters.

(or maybe vendors and consultants invented the term to make more money idk? believe what you want)

1 - SDLC / S-SDLC differences and challenges

1.1 - SDLC

Again, people in a certain meeting maybe illuminati included, they decided to make the security checks in a cycle as in software development. (SDLC) Before security steps included, the software development lifecycle looks like this;

I - Planning

This step is all about "what we need, what should be done to be succesful" with few extra steps and some business people involved.

II - Analysis

In this step, the project's scope should be defined. Again, mostly business analysts and product owners are involved. Also developers are coming into stage at this step to give feedbacks too.

III - Design

Business people are closing their outlook clients, and the developer team starts to work on documentation, HLD and stuff.

IV - Implementation

Simply, the step is mostly writing the code base. (brutally simplified version)

V - Testing & Integration

Boundary value analysis, Equivalence Partitioning and other bug finding techniques are being done in this step.

VI - Maintenance

Deployment is done and regular health checks are performed and the environment is monitored.

Traditionally, a lot of security tests are being done after the deployment. I mean, usually the applications are tested at their final version. The big problem is, when sometimes critical security flaws found, it is tough to fix because of these reasons;

* The fix requires a design change.
* The fix requires a change in a functionality which is used by a lot of parties.
* The fix requires to alter the database (columns/connections or else) that is already in-use.
* The use-case is designed in such a way that there is no provision for misuse or exploitation. (naturally vulnerable?)
... and more

You -as a security guy in your organization- are late because you waited too long for the final version of the product in all this fast-paced feature-driven development cycle (consumption frenzy but some people call this agile), and you innocently decided with the developer team that many of the product's functionalities were not ready and security tests could not be done properly. Hold your tears, this is expected and happens to all of us, all the time.

1.2 - Secure SDLC

This model brings project teams and security teams together at the table and work together from the very beginning. First and second step mostly are the same but other steps are altered like this; (second and third steps can be combined for the security subjects)

III - Design

This step now includes Threat Modeling and Security Design Review. One person from security team will also be joining the designing meetings and points out the threats that project has, and gives feedbacks for the network topology, HLD. The person decides which security product should be positioned, which data should be stored, how it should be stored, which transactions should be logged, which security tests should be performed and when.

IV - Implementation

This step requires some automated security scans. These scans must be done for every time code-base is changed. This requirement assumes that the product has a CI/CD. Which you could squeeze and fit some security products in those pipelines. (hope already have) Relative security scans for this step can be SAST, Secret Scanning, Container Security Scan, SCA, and all the scans that can be done without requiring the application to run. We will deep dive on these topics in further, for now all you need to know these scans are related with this step.

V - Testing & Integration

This step now involves the scans that can be done only with interaction the application itself. Some teams also prefer to do penetration tests every cycle but you have to be really rich to coordinate this chaos and have the manpower if your organization is that big. (manual effort is your enemy) Other well-known automated security scan type is DAST. Although DAST scans are not effective as penetration tests, these scans allow you to find misconfigurations that can be overlooked and vulnerabilities that can be easily exploited. Fine-tuning and integrations are headaches and even sometimes you can feel like you will never be able to scan the applications properly.

As the title suggests, test environments are perfect for building these automatic scanning scenarios. The main goal we want to achieve here is to test the application only. When you perform those automated scans in production environments, other products and solutions in between (API gateways, WAFs or unified network products, rate limiting, the steps cannot be automated, like OTP) become an obstacle for you at this time.

VI - Maintenance (or Deployment)

Security alerts and logs are being monitored. Agents are up and scanning the servers/configurations and all stuff together. Vulnerability Management is also here. (ofc it depends how you define what VM is, lets say more or less)

1.3 - What changed?

With this Secure SDLC model, you crushed two process into one and now security operations also aligned with the development phases, walking together. The development and business are happy now and the applications are being fixed in time, right? Absolutely not. Your scans are breaking their CI/CD pipelines when some security issue is found, security agents are consuming resources on application servers, plus if these agents are configured incorrectly or a problem occurs, it may even lead to a service outage. Now you have a more tense environment than ever before. Fast-paced cycles are now even more hostile to security teams because they often brag about delivering on time, and you make them fall behind on their goals by having to fix something in every sprint. Do not be afraid, love begins with hate tho. These concealed feelings define what a team is. Since the security team is now someone you see/talk/discuss frequently, you are away from the tone of "you must do these things, otherwise we wont let you to go live!!!!!" that is imposed by completely different authorities.

The key word is "together".

2 - Conclusion

In conclusion, the consumption frenzy era has dragged us headfirst into the world of DevSecOps. If you are pushing new features every other week (or every other hour, if you are feeling particularly wild), you can not afford to slap security on at the end like some flimsy sticker. Transitioning to automated tools is not a magic wand -— it is survival. Sure, it is a love-hate relationship. You shift left, catch more issues early, and boom! More mails from devs how much they love you and your issues. (what?) But hey, is not that the beauty of working together? In this feature-addicted world, DevSecOps is the awkward roommate you did not ask for but absolutely need.

In the next blog, I will show you how to simulate this chaos at home. We will set up a CI/CD pipeline without the "Sec" part, so you can experience the glorious headaches yourself.

That is all for now! Thanks for finding my blog interesting enough to read. Have an amazing year (just in case I do not publish the next part before 2025), you absolute legends! 🎉