Posts

Decision Tree with Python

Image
Introduction Last time we talked about non-linear classifier using Support Vector Machines or SVM . Today we'll be discussing another non-linear classifier and regressor called decision tree. The way decision tree works is by creating a model, which predicts the value of a target variable by learning simple decision rules inferred from the data features. Since trees can be visualized and is something we're all used to, decision trees can easily be explained, visualized and manipulated the non-linearity in an intuitive manner. Surely there are some disadvantages as well, but we'll note them a bit later, firstly let's see them in action. Implementation It won't come as a complete surprise to you, that scikit package has already taken initiative and implemented the whole thing using DecisionTreeRegressor and DecisionTreeClassifier classes. What is left for us is to bear the fruits of someone else's hard labour. import StringIO import numpy as np im...

JavaScript Adapter Design Pattern

Image
Today we'll continue the JavaScript Design Patterns series by discussing the Adapter Pattern . Adapters are added to existing code to reconcile two different interfaces. They allows programming components to work together that otherwise wouldn't because of mismatched interfaces. Adapter may be also used to ease the use with the existing interface. If the existing code already has an interface that is doing a good job, there may be no need for an adapter. But if an interface is unintuitive or impractical for the task at hand, you can use an adapter to provide a cleaner or more option-rich interface. Let's see how it looks in the following illustration: Here we depict the example of legacy  IDataManager interface, which is deeply used within the system. With the introduction of Redis database into the system, we need an adapter to fill the gaps. Our Adapter class has to implement the getData method, to make it consistent with the system, calling in turn the scan ...

Support Vector Machines with Python

Image
Introduction Having learnt different optimization and classification methods, you must feel quit confident to start exploring the datasets of interest. However you might very quickly run into a dataset, which no matter how hyper-parameters are set , is just not linearly separable. Surely there is no place for despair, especially since we have just a classifier to deal with these situation, called Support Vector Machine. How it works Support Vector Machine, or SVM, are a set of supervised learning methods used for classification and with a slight change for regression. The core idea of it is to linearly separate the hyper-space of features. The prefix hyper is not occasional, as SVM increases the dimension of feature space to achieve it's goal. The power of the method comes from using kernel functions , which enable it to operate in a high-dimensional, implicit feature space without ever computing the coordinates of the data in that space, but rather by simply computing...

Proof of Work (PoW)

Image
Proof of Work (PoW) is a central part of cryptocurrency and blockchain technology. But what is Proof of Work (PoW) and why is it so important to cryptocurrency? Here’s everything you need to know about the protocol that gave us Bitcoin (BTC). What Is Proof of Work? Proof of Work (PoW) is a protocol designed to make digital transactions secure without having to rely on a third party. Essentially, PoW requires members of a community to solve challenging puzzles. This work builds on previous puzzle solutions. As a result, PoW is a way of verifying current and past transactions. Additionally, the work that goes into solving the puzzle generates rewards for whoever solves it. In the world of cryptocurrency, that’s basically what “mining” is. Proof of Work (PoW) is a foundational concept for anything having to do with blockchain. Background The concept behind Proof of Work (PoW) was originally invented by Cynthia Dwork and Moni Naor. More specifically, they explained the i...

JavaScript Promise

Nothing weights lighter than a promise This maybe true regarding to human promises, however in the programming domain, promises are always kept. Following this optimistic note, today we'll be talking about JavaScript promises. Event Handling Problem Let's see what promises are good for and their basic capabilities starting with a problem they come to solve. Events are great for things of a repetitive nature like keydown , mousemove etc. With those events you don't really care about what have happened before you attached the listener. On contrary calling services and processing their response is a completely different kind of beast. Have a look at the following function, which reads a json file and returns it's content or an error in case of something goes wrong. function readJSON(filename, callback) { fs.readFile(filename, 'utf8', function (err, res) { if (err) { return callback(err); } try { res = JSON.parse(res); ...

Operation Timeout in MongoDB

Today I'd like to talk about a problem every MongoDB developer should be aware of - operation timeout. I have surely risen a lot of eyebrows and a few snide remarks, but let me reassure it's worth reading. Connection vs Operation Timeout So where do we start? The main problem with operation timeout in any database, not specifically to MongoDB, is the developer's confusion between connection timeout and operation timeout. So let's clear the air right away by clarifying the difference. Connection timeout is the maximal time you wait until you connect to the database. Whereas operational timeout is the maximal time you wait until a certain operation is performed, usually CRUD . This happens after you're already connected to the database. Post MongoDB 2.6 If you've just started using MongoDB or had a luck to upgrade your existing instance to the newest version, that being 2.6 at the moment of writing, then you should know there is a build-in support ...

Dimension reduction with Python

Image
Introduction Sometimes, no matter how good an algorithm is, it just doesn’t work. Or worse, it doesn’t pick up anything. Data can be quite noisy, and sometimes it’s just about impossible to figure out what went wrong. It's worth noticing that the most interesting machine learning challenges always involve some sort of feature engineering, where we try to use our insight into the problem to carefully craft additional features, that the machine learner hopefully picks up. Garbage in, garbage out, that's what we know from real life. Not surprisingly this pattern also holds true, when applying machine learning methods to training data. To tackle the issue, we will go in the opposite direction with dimensionality reduction involving cutting away features that are irrelevant or redundant. There are several good reasons to trim down the dimensions as much as possible: Most of the models hate high-dimensional spaces and superfluous features, which often irritate or mislead th...