MySQL vs PostgreSQL is one of the never-ending wars in the RDBMS world. This article will help you to choose the best fit for your use case.

Source — silota


Recently, during a project I was working on, we were given a requirement:

— “To retrieve records from the DB within 10 milliseconds consistently, from two large data sources/tables.”

One data source contained approximately 686 million records and the other contained 24 million records.

On top of this, the requirement stated that the preference for this solution should be towards an AWS managed DBMS, so the database instances can be easily managed in the long run.

Thus, with these requirements, the hunt for the best-suited DBMS began. 🤞🏽

Why Amazon Aurora?

With the data volume being huge, and the read rate high (about…

As the first post of this analysis series, I’m analyzing overall crime rate change, House Break and Theft in Sri Lanka during 2010–2018.

Image by Gerd Altmann from Pixabay

Data Collection and Cleansing

I have collected the crime data of past 8 years from the official Srilanka Police website: Police crime statistics

As all the data are in PDF format it is converted into CSV using python Tabula library. Then missing data and corrupted cell values during the PDF to CSV conversion were fixed with excel.

from tabula import convert_intoconvert_into("distribution_of_grave_crime_abstract_for_the_year_2014.pdf",        "distribution_of_grave_crime_abstract_for_the_year_2014.csv", pages="all")

Overall Crime Rate

Even though the Population of Sri Lanka is increased by 7.278 %, the overall crime in Srilanka over the past 8 years is reduced by 36.8398%. That’s great news for all Sri Lankan Citizens. …

Python downloads its dependencies from PyPI repositories by default. It contains latest versions (can be stable or not) and various amount of packages. We’re good right? So, whats the need of custom private package repository?

Dependency management

You can control dependencies of packages explicitly regardless of versions deprecation or latest backward incompatible versions. Of course, this can be done by defining versions in requirements.txt, but to make sure that every package we get is as we expected its better to use custom repository.

TLS v1.1, v1.0 Deprecation by PyPI servers

PyPI servers have discontinued support for TLS 1.0 / TLS 1.1 version devices to download packages.

“I am going…

The syslog-ng application can read messages from the sources. It processes them with filters, rewrite rules, parsers and finally sends messages to their destinations. The syslog-ng application already has a Kafka destination that is implemented in Java. The Kafka source will allow syslog-ng to read messages from Kafka. This can be used for example as a queue between several syslog-ng instances.

Syslog-ng : Kafka source

Kafka source is my Google summer of code project for the year 2016 with the Syslog-ng organization under the guidance of Viktor Juhász. Syslog-­ng can read messages from the sources. It processes them with filters, rewrite rules, parsers and finally sends messages to the destinations.Syslog­-ng has a Kafka destination, which is implemented in Java. Reading messages from kafka will make it as a queue between two Syslog-­ng instances.

Kafka is a distributed messaging system provides fast and highly scalable and redundant messaging through a pub­sub model. It allows a large amount of ad­hoc consumers and also it self heals from…

It’s my third and final project during my internship at WSO2. It is a property of software that enables a system to perceive that it is not operating correctly and, with/without human intervention, make the necessary adjustments to restore itself to normalcy. I read a few research papers since this area is still new to the industry, I couldn’t find any actual implementations. Therefore, I had to spend a lot time reading research papers to get an idea about self-healing software.

Fault tolerance computer systems mirror all their operations, so if one component fails another redundant component will take part…

Vithulan MV

Computer Science Engineer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store