By Omolola Omolambe

Introduction

The Nigerian Fintech ecosystem is filled with over 200 Financial Technology startups, with new startups being minted ever so often. A significant surge was seen between 2020 and 2022, known as the FinTech boom, where the funding for the sector was over $507m, a substantial growth over previous years. The Nigerian Fintech sector is the most funded technology sub-sector, and its services vary from providing access to financial services to payment solutions and investment solutions.
Despite the impressive growth in this sector with different offerings, Fintech solutions still represent only a tiny portion of customers’ wallet shares as there is still a lack of trust. This is a three-pronged problem: lack of trust in scalable infrastructure (noticed through frequent outages), lack of a physical presence (a necessity for customers because of the failed companies that have vanished from the map), and what can be interpreted as the capricious nature of the CBN.

My goal with this article is to discuss the one item an organisation might have control over here, the CBN, Iike the scalability of your infrastructure.

Scalable Engineering Systems and Their Importance

Scalable Engineering Systems are systems that can handle an increase in simultaneous users/customers and their needs, including requests, data, or transactions. Scalability is critical in technology organisations, especially when money is involved, as one “glitch” can prevent return customers and spell doom for your company. It is also crucial that scalability is achieved in the most cost-effective way to ensure that the cost of retaining a customer is below the cost of acquiring a customer, as the CAC (Customer Acquisition Cost) should be the highest cost incurred in the customer journey.

A Software application built with non-scalable architecture could function at lower request demand, so it can be hard to determine if it can handle growth initially when the workload is light. Unscalable applications can still perform well under low-utilisation conditions, making this issue less obvious. With a limited number of simultaneous users, the application will handle the demand and the incremental user cost might take time to notice or adequately measure.

Scalability is crucial in the financial sector because of the sensitivity towards where money is kept. As more data is stored or more users access the software simultaneously, its architecture becomes more strained. Performance can decline significantly if this demand exceeds the software’s capacity to scale.

In a low-trust environment like Nigeria, downtimes or slow requests can cost businesses more because users quickly abandon the product and withdraw their money which becomes tantamount to a bank run and could cripple the company. This type of unreliability increases the potential for user churn.

Change is a constant factor in business. Companies often need to pivot, and building scalable software typically makes it easier to add new features over time. This approach requires less development time in the long run than unscalable software. As a result, businesses benefit from software that can adapt quickly and incorporate new features to remain competitive in the market.

Navigating scalability across different stages of business

The company’s stage might determine how much emphasis should be placed on scalability or the type of product (internal or external). For an early-stage startup, scalability might be considered but not prioritised above finding the product market fit and going to market. For mid-level or big tech companies, scalability might be more critical because the product is at a more mature phase, and it’s crucial to prioritise retaining customers and growing the business.

Building products in large tech firms and big corporations may take months, depending on the type of company. In contrast, startups are often required to achieve similar developments within weeks, which makes scalability more challenging during those periods. This struggle is significant because products and systems must be as flexible as possible, allowing them to adapt quickly as the company grows into a medium-sized entity. This pressure can make it difficult for the team to prioritise effectively. Additionally, the type of product being developed can influence priorities; for example, a small prototype may focus on qualities other than scalability. Internal software designed for small businesses, which often serve a limited customer base, might emphasise different factors over scalability.

Scalability is generally more straightforward and requires fewer resources when considered from the architecture planning phase.

We have experienced frequent banking service downtime in Nigeria, lasting from a few hours to several days. Traditional banks can typically recover from these disruptions without significant user drop, but startups are disproportionately affected due to low user trust in these institutions. These disruptions also significantly impact users’ lives. For example, a user might be unable to access their funds to pay for emergency medical services, or a student might be unable to transfer fees before a deadline. The consequences for both customers and the company are severe.

Best Principles for Building Scalable Engineering Systems

Designing scalable systems requires a deep understanding of the architecture, infrastructure, technologies, algorithms, and business requirements. A fundamental reason for scalability issues is a buildup of queues within the system. To make your system scalable, several critical aspects come into play.

Architectural Design and Decisions

A good understanding of the business requirements should inform how a system is built. This means a clear product document should exist. Therefore, it is crucial to consider how easily the system can accommodate changes when making design decisions. These decisions include how to handle requests – synchronously or asynchronously, what type of database to use, what kind of service architecture to use, etc.
Your application’s requirements should determine whether you choose a monolithic or microservice architecture. Scaling for specific use cases may be difficult with a monolithic service, but it becomes easier with a microservice architecture. Examples of design decisions: creating multiple clones of the transfer service allows you to scale it based on demand. In contrast, the report service may only need a few clones if usage is typically lower.

Code Quality

Every software engineer should write code considering the next person who will maintain or use it. Code quality and cleanliness should be the other considerations after getting your code to work. Even if functioning code is written, it is essential to consider whether it is the most efficient way to handle it in the long run. Scalability involves writing code that addresses current problems and allows for future growth with as little or no hindrance as possible. This doesn’t preclude technical debt but should be avoided when considering growth. Avoid the “spaghetti” code.

Database Operations

When writing queries, it is necessary to consider the efficiency of the “queries” and when to apply indexing. Archiving old data from the database can also help improve scalability. For instance, in a university, the records of students from beyond 5 years ago may be accessed infrequently. They can be archived or kept in storage that is not as expensive, keeping the records for active students easily accessible and easier to reach. This allows for a reduction in the size of the database and the cost of storage while improving data retrieval performance and, consequently, user experience.

Caching

Caching helps reduce the overall load on your application. When you frequently request static data (e.g., states in a country), caching this data helps deliver the data faster without communicating with the primary application storage. Understanding which data should be cached is essential to avoid taking up unnecessary space.

Asynchronous processing

Identifying tasks that can be performed without requiring the user to wait for their completion—and converting them into asynchronous operations—can significantly improve system scalability. This approach reduces API latency for users and distributes workloads across separate threads, improving overall performance. Techniques such as using multiple threads, message queues, or an event-driven architecture can help make this process possible. For example, when a user purchases, the system can confirm the payment immediately while sending a confirmation email asynchronously. This way, the primary objective of processing the payment is achieved efficiently, and the secondary aim of customer record retrieval is handled asynchronously.

Examples of best practices in the FinTech Space

1. Architectural Design and Decisions: A payment gateway that manages merchant transactions might utilise a microservices architecture in which the payment processing and reporting services are separated. During peak traffic periods, such as Black Friday, the payment processing service can be scaled up to accommodate the increased transaction volume, while the reporting service can maintain its normal capacity. This approach allows for independently scaling critical services based on real-time demand, ensuring optimal resource utilisation and cost efficiency.
2. Code Quality: Consider the transfer function in a digital wallet application. Poorly structured code might consist of a single large function that handles all transfer types, such as wallet-to-wallet and wallet-to-bank transfers. This code could be cluttered with multiple nested if statements and duplicated validation logic. A better approach would be to create separate, well-named functions for each validation step and transfer type. For instance, a transfer can be broken down into clear steps: validateBalance',validateTransferLimit,processTransfer, andnotifyUser`. This structure makes the code easier to maintain, test, and modify when adding new transfer types or changing business rules.
3. Database Operations: Effective transaction record management is crucial in a payment system. One simple yet effective approach to optimising the database is to index frequently searched fields, such as user_id, transaction_status, and transaction_date. For instance, when users want to view their transaction history, the system often searches by user_id and filters the results by date range. Creating appropriate indexes on these fields can significantly speed up these common queries. Additionally, implementing a basic archiving strategy—by moving transactions older than six months to an archive table—can help reduce the size of the active transaction table. This reduction leads to faster daily transaction queries while maintaining data access.
4. Caching: In a digital payment platform, certain data elements are accessed frequently but are rarely changed. An example is merchant configuration data, which includes accepted payment methods and transaction limits. This data is read during every transaction but is typically updated only when configuration changes occur. Implementing a distributed cache for this data can significantly reduce the load on the database and improve transaction processing times. When a transaction is initiated, the system retrieves the merchant’s configuration from the cache rather than the database. The cache is only invalidated when there are setting updates, ensuring consistent performance.

Resource Scaling Options

There are two types of scalability to consider:

1. Vertical Scaling—Vertical scaling involves adding more resources to a single device, such as increasing the storage capacity of a flash drive. For instance, if you have a 32GB flash drive and want to store more files, you can replace it with a 64GB flash drive. Vertical scaling usually hits a hard limit or becomes untenable from a cost perspective. At some point, you will reach the maximum capacity that a single drive can handle.
2. Horizontal scaling: Horizontal scaling can be represented by using multiple flash drives. Instead of upgrading to a larger capacity drive, you can use various smaller drives to handle the increased load. For example, you can use two 32GB flash drives instead of one 64GB drive. It is more flexible because you can add or remove flash drives as needed to handle varying workloads. This is where elasticity comes into play and refers to a system’s ability to scale resources dynamically based on current demand; horizontal scalability makes this process easier. Many cloud providers offer these functionalities, automatically allocating additional resources when demand spikes and releasing those resources at lower demand levels.

Conclusion

Understanding these principles and knowing when to apply them to your specific use case is essential. Using a one-size-fits-all approach can lead to significant problems as there is no magic bullet. Each application has unique requirements that call for customised design solutions. Scalability is not a one-time task; it is an ongoing process that requires continual monitoring and analytics to gain insights into your system’s architecture and user behaviours. Tools like New Relic can assist in providing the necessary monitoring to address issues proactively. Scalability spans the entire lifecycle of a system, from design to deployment and maintenance. It’s important to consider scalability from the initial planning phase rather than treating it as an afterthought. Therefore, it demands ongoing attention and refinement to ensure that as your system grows, it continues to meet the needs of both your users and your business.

*Omolola currently leads an engineering team as a Senior Software Engineer.