Building a large-scale system is a much bigger job than building a simple application in an IDE like Visual Studio. One person can do it alone, but they have to do everything themselves, and that is a much harder road. A development team splits the work, and each developer's contribution compounds, so the system gets built faster and stays easier to maintain. Either way the work demands real depth: software engineering practice, programming, databases, information security, and networking and cloud computing too if the system is online.
An enterprise system running a business and its objectives is a good example of a large-scale system. Whatever the size, every software system needs a procedure to follow. A developer who skips planning and starts coding straight away will end up with a system that fails. That procedure is the System Development Lifecycle, the SDLC, and its activities are Planning, Analysis, Design, Implementation, Testing, Deployment and Maintenance. But the SDLC only gives you the structure. It does not tell you how to carry those activities out. That is what a methodology is for.
Structure comes from the SDLC, rhythm comes from the methodology
Agile is the most popular methodology, and it says the SDLC activities should be repeated rather than run once. Those repetitions are called sprints. A sprint is a fixed timebox, usually one or two weeks, and inside it a slice of the system goes through all the activities and comes out usable. The sprint ends when the timebox ends, not when a task does.
One sprint does not build the whole system. It builds a small, manageable part of it. Building the whole thing takes many sprints. Waterfall is the opposite. Each activity happens once, in sequence, and the system is finished and ready for users only when the last one is done. The difference matters more to the customer than it does to us. Under waterfall the customer finds out whether we built the right thing at the end, when changing it is most expensive. Under agile they find out every sprint.
Architecture is the set of decisions that are expensive to change later
Software architecture is the overall design and structure of the system and how it is organised. It is the output of the design phase and the input to implementation. It is shaped by the functional and non-functional requirements, and it carries the database design, the visual diagrams and the rest of the planning. It also names the stakeholders, the people with an interest in the project.
The reason architecture deserves its own attention is that it is where the expensive decisions live. Most code can be rewritten in an afternoon. How the system is split up, where its data lives and how its parts talk to each other cannot.
Planning: the customer is the specification
Planning is the first activity. Interviews and customer feedback are an important part of building a system that meets the customer's needs. They get documented as use cases and then turned into functional requirements. Functional requirements are the things the system must do. For example, "Users must be able to login with their username and password."
Non-functional requirements cover Functionality, Usability, Reliability, Performance and Supportability, plus constraints. FURPS+ is the usual shorthand. "The login process must happen fast, within one second" is a non-functional requirement, because it is about performance rather than about what the system does. Getting this right is not paperwork. It is the cheapest moment in the whole project to discover that the customer wanted something else.
Analysis: break the system into parts
Analysis is about taking the system apart into its components. Functional and non-functional requirements are identified, analysed and documented here. The system analyst is responsible for delivering the work in this phase. Stakeholders can also be identified in this phase just like the planning phase does. The work that the system analyst must do, can involve writing Use Cases, Use Case Descriptions and Use Case Diagrams. A Use Case is a specific goal or function that a user wants to accomplish. System Analysts use techniques that are the User Goal or Event Decomposition technique to identify these use cases. Once the use cases have been identified, it can then be transformed into functional and non-functional requirements. A Use Case Description is a detailed view of the use case. A Use Case Diagram visually represents the use cases that includes the actors and the interactions with multiple use cases.
Implementation: foundations first
Once planning, analysis and design are done, the developers implement the architecture and build the product. They start with the foundations: create the project in the IDE and set up the database. The database developer works from the database design, which includes the UML entity relationship diagrams and the data dictionaries. The programmers work from the UML class diagrams to understand how to structure the code. Between them they write code that satisfies the requirements. A functional requirement like "Users must be able to create an account with the system" is not finished until a programmer has written the code that makes it true.
Testing is not a phase you can skip
Where testing sits depends on the methodology. Under waterfall it is a phase near the end. Under agile it happens inside every sprint, which is one of the reasons agile finds problems while they are still cheap to fix. Either way the system must be tested properly before release, and the bugs that are found must be documented so the programmers can fix them. Once development and testing are done the system is deployed and customers start using it. Anything that surfaces after that is handled in maintenance.
My project that I did at university
When I was in my final year of my bachelor’s degree, I worked on a group project that required us to deliver a mobile application to a real customer. We had to follow the full SDLC cycle and we chose the Agile methodology. I was responsible for designing and developing the mobile application and I focused heavily on the Design and Implementation phase of the SDLC. The other members of the group communicated with the client and they told me what I must do based on the feedback.
The project required us to mimic the company’s website as a mobile application. I designed the user interfaces for both smartphone and tablet devices. The project was an Android application, written in Kotlin, and it was a large-scale system to meet the customer’s needs. This project required me to know about networking, cloud computing and databases. The end result of the project was satisfactory and the customer was pleased with the project that our group delivered.
Online systems bring networking and the cloud with them
If the system is online, or connects to something online, the developer needs to understand networking and cloud computing. A web application, a desktop application that needs an internet connection, a mobile application backed by an online database: all of them mean HTTP requests and responses, hosting decisions, and usually a cloud platform holding the database.
Security: what the code must do, and what the platform must do
Information security is critical, especially on a large-scale system with many users and a lot of data. A threat actor is someone who tries to break into a system to steal user information or to do the system harm.
It helps to separate the defences into two layers.
What the code must do. Encode output and never trust input, which is what stops cross-site scripting. Lock accounts or slow them down after repeated failures, which is what blunts brute force and password spraying. And never store passwords as plain text. Hash them with a slow, one-way algorithm built for the purpose, such as bcrypt or Argon2, with a unique salt for each user. Hashing is not encryption. Encryption is designed to be reversible. Hashing is designed never to be.
What the platform must do. Rate limiting, a web application firewall, and a content delivery network in front of the system. Denial of service is rarely something you can code your way out of, because the traffic never reaches your code in a shape you can reason about. It is absorbed at the edge or it is not absorbed at all.
Scale and design for failure
As more users arrive the system has to scale: more storage, more compute. On a cloud platform that is a configuration change rather than a purchase order, which is a large part of why we host things the way we do.
Large-scale systems also have to be designed for failure, so that when something crashes the system either keeps working or recovers afterwards. Server redundancy is the classic example. If one server stops responding the application carries on, because the others are still there. The same instinct shows up as health checks, retries, spreading instances across availability zones, and degrading gracefully instead of failing completely.
Designing for failure is really a statement about the customer. It says we would rather the system be slightly worse for everyone than completely unavailable for anyone.
UI and UX decide whether any of this was worth it
User interface and user experience design have to be considered from the start if the project is going to meet the customer's needs. Customers are more satisfied with a system that is fast, responsive, easy to use and modern.
UX overlaps with non-functional requirements, because it speaks to performance, usability and accessibility, and it can touch functional requirements too. "The user must be able to login within one second" is a non-functional requirement, and it is also a UX requirement. The two are not in competition.
In closing
Building a large-scale system is a challenging task that requires the team to deliver a product or service that needs to satisfy the customer’s needs. You could spend months to years building this system and it could still fail. Don’t expect every project to be a success, only expect it to be a success if you gave your best effort at it. That is what leads your team to be a success on what you deliver to customers. They need quality more than quantity.
About the author
I am Matthew De Waal, and I work at First Technology Digital Cape Town as a software developer intern. I contribute work by writing code for the backend and frontend of systems. I recently graduated with distinction at IIE Varsity College and I enjoy what I do in my career. I am determined to learn more about Computer Science.
