TL;DR
there is a recent thread with a topic something like "What process do you use to write programs." This thread seems to be overlapping with that, which is fine with me. Out the the procedural programming world, there are many approaches: waterfall; test-driven design; agile/scrum/many other names; etc. This will be a post about waterfall vs. agile.
As understood by most people, waterfall at its core is based on requirements; it is the traditional way software was done back in the day e.g. similar to building a process from a design specification. The relevant parties look at the process, and define what the process must do, along with maybe some items it "should do." A lot of effort goes into the requirements document, then it is passed on to the programmers. The theory most people have about waterfall is that you go through once the requirements document process, then once through the programming process, and when the software meets the requirements (e.g. write tests), the system will work. I was hired late on a project as a remote worker. I would get on telecons and all they would talk about was section this or that section of the requirements document, and I remember thinking, "is anyone writing code?" There was a critical design review (CDR) scheduled that would determine whether the code passed a set of tests, and the project was producing so little, after three years, that the pass/fail metrics for the CDR were dropped back to a PLC/process equivalent of "if a box of a particular size is placed on the end of the conveyor, the prox senses it and the conveyor turns on." So the CDR starts, and they are failing 80% of the metrics. This one test engineer, she saved the project by standing tall and saying "this failed" and not allowing exceptions (e.g. coders will say "yes it failed, but we found it was a single typo in line 1356 and it will work now," and sometimes that is accepted in place of a successful test). Oh yeah, and the conveyor did not start when the box hit the prox. So maybe a third of the way through the two-day CDR, the head honcho shuts the CDR down and starts interviewing people about the project. Remember, this has been going on for three years.
Some people get fired, new people are brought in, and the new head programmer switches to agile. He asks the young programmers "should we refactor?" and they give a frightened, confused, "yes?" Apparently his predecessor more or less wielded the requirements document like a metaphorical (not physical) club "make the code do what it says here and the system will work," and everyone was abused and traumatized.
So they switch to agile. Agile, to me, is defined by small bites and humility. Small teams work only on one small piece of the project for small periods of time - two weeks is long. Then everybody gets back together and reviews what was done and whether to accept it into the project. The coolest thing about this process is, in that environment, when everyone is in the room looking at proposed code and asking questions, it is most often the author of the code who ultimately decides to reject that code in its current form during its review, and often for a very small thing that they know can be better. That is one place where humility comes in.
The other place for humility is in the overall approach, and takes a bit to explain, and for that here is a common example of how an agile process works. An agile-based software company comes in to do a project, and the client says "here are our 100 requirements." the agile folk say "What are the most important three?" The client says "There are 100, and they all must be met." "Yes, we know, just identify the top three." So the client points out the top three, and Agile heads off to code them up. They come back in a week, and review them with the client, find where any tweaks are needed, then ask "what are the next three most important?" "But there are 97 more" is the response, and you can guess how this is going. But the punch line is, after Agile completes about 20 items and asks for the next three, the client says "Don't bother, the system is fine as it is now, you're done."
So the humility in that story is that the agile-minded assume they do not fully understand the process, and work on a small piece they think they might understand, and the process knowledge gained from coding each small piece builds up and informs the design and coding of the future pieces. Another aspect of the humility is the willingness to refactor, literally throwing away the code to that point and starting over, when they see it is not going to work.
On the project I was on, they refactored the overall design three times at the beginning, and after a few weeks they were halfway to the box/conveyor/sensor problem (that was a metaphor for the current audience; it was more complex than that) that had not been solved in the previous three years, and about a month later they were well past that and they have not looked back. Looking at the system they have now, I can see analogies to automated processes, SCADA, history, and the ability to go back automatically detect changes in a spec and regenerate the relevant products.
One interesting sidebar is that, while the most of the world treats waterfall as a once-through process and expects a working system at the end of it, I heard that the person who came up with the waterfall process says it needs to be run two or three times or more to get get final system right. So in one sense, properly run waterfall as lot in common with agile, it is only the sizes of the bites taken that differ.
P.S. I have greatly oversimplified "agile" here, There are scrums and scrum masters and product owners and a lot of other pieces. I expect some would disagree with my characterization, but that is what I saw and it was amazing.