Machine learning and statistics with python

Introduction: I go through different things people comes searching to my website and recently I found out this new term named ragged and jagged list. I wasn't familiar with the term, so I read about it and going to talk about it from different language and usage perspectives in this article. So by the end of this article, you should be a master of the concept called "ragged or jagged array". Definition: A ragged array, also called jagged array, ( yes they are two same things), is basically a non-uniform array, which consists of different length lists as elements. For example, [[1],[1,2,3],[2,5]] is a jagged array. The name arises from the concept that if you list the elements down one by one vertically, then the edge will be ragged/jagged. How to create: In python, it can be quite simple to create. You can just take an empty list []; and then keep pushing different lists one by one using append. For getting a better idea, read about python lists . For java, read it f...

Analytics vidhya May JobAthon: EDA and solution Written By: Saurabh Shelar  Introduction:   Nearly a month ago I had participated in Analytics Vidhya JonAThan May 2021, where they had given us data and I was supposed to analyze it and make a predictive model in 3 days which I couldn't do it in time since I was not getting the desired accuracy, so I decided to take it as a challenge and pursue it as one of my practice projects. Problem briefing:   The data I was using was about customers of a bank called Happy Customer Bank, which is a mid-sized private bank that deals in all kinds of banking products like savings accounts, current accounts, investment products, credit products, among other offerings. In this case the bank wants to cross sell its credit cards to its existing customers. The bank has identified a set of customers that are eligible for taking these Credit cards. The customer’s details included their age, gender, region and details of his/her relationship...

Introduction: A few days ago, I got a model called xyz.model. We got it from a client and I had no idea how to open it. And after searching a lot, I couldn't find out an useful answer though. That's why this article. What is the object? a .model model object is created on saving xgboost model from R, using xgboost::save() method. That is how it is created. Now as you may guess, using a R model in python we will have a problem. And you are right. In general the standard xgboost library will not be able to load according to  this github thread . What is the issue? Actually it is impossible to load a model from a buffer (e.g. bytestring) in Python, due to a bug where ctypes is used incorrectly to get a const char* pointer: >> > bst . load_model ( b'123' ) TypeError : underlying buffer is not writable Here's a short example of what it is basically trying to do: >> > b = b'123' >> > ( ctypes . c_char * 3 ). from_buf...

Problem: Often, you will need a single file to download from github, but you can't figure out how to do that. In such a case, a lot of people will suggest that you will need to download the whole repository. But that's not a useful suggestion and you know it. So what to do? Solution: In such a case, first click the view raw option in github ui. This will redirect into a new tab generally and show the rawgithubusercontent.com/... site and the raw content in that tab. Avoid this. Get this raw content in the same tab. This will create a similar rawgithubusercontent.com/... url.  Now, using this url, use curl <url> to fetch this content. This will download the raw content in your computer.  Installation (if needed): For linux and mac, you will have this curl installed by default. For windows, you need to install it if your version is not later than windows 10. To try out the installation see Testing your cURL installation below. [the following part is taken from ...

 Introduction: Hi! today we are going to talk about a different and simple topic which I encountered recently in an interview. The topic is csv. As a data scientist, we use pandas daily to read csv files using pd.read_csv() functionality. And that is where the csv turns into a dataframe for us and we start data manipulation with the dataframe. But a csv is not a dataframe, and doesn't need pandas to be read. In this article, we will discuss a few ways to read a csv file, and will also take a deeper dive in how the csv and similar files are actually stored. What is csv? A csv or comma separated value file is a file where each data item is stored with comma separating one value from another. csv is the universally accepted format for tabular data. For a normal csv, each row is stored as a separate new line in the file; and each row contains each values separated by comma, but the row doesn't end with a comma.  How can we read a csv? A csv can be read using pandas ( read_csv...

Introduction MongoDB is a noSQL database and in this document, we will be working through a datacamp course of introduction to Mongodb and provide a brief introduction to mongodb. This is the first part of the series of post where I summarize the findings from the datacamp course. what is NoSQL? NoSQL stands for not sql; i.e. non-relational databases. Including MongoDB, graphql and other databases, non-relational databases are results of big data and non-relational data models which model the data in a relation less manner. the common ways are graph node-edge systems, key-value pairs(mongodb falls in this) and others. NoSQL are good for scaling up with huge load of data and they store data in non-tabular formats unlike sql and other relational database systems. MongoDB basics: What is MongoDB? Mongodb is one of the free, nosql database systems; which is scalable from its core, and provides a json format data to easily interact, query and plug with your applications. How does a mon...

 Introduction: One of the best methods of speeding up your python codes is turning it into a cython code. For turning a python script to cython, you need to save it with a pyx extension and then compile it using a setup file . For example, say you have a file named helloworld.pyx. To compile this, your code inside setup file will look like: from setuptools import setup from Cython.Build import cythonize setup ( ext_modules = cythonize ( "helloworld.pyx" ) ) Now, after doing this, you have to run this setup file using the following command which creates a .c and a .so file. $ python setup.py build_ext --inplace The Problem: So even after compiling, you may end up getting: importerror:... undefined symbol: _py_ZeroStruct when you try and import the compiled cython module.  Why does it happen? and the solution: the source of this problem, according to the stackoverflow is the mismatch between the python version of the compiled cython code and the python f...

Introduction: This week, I took part in a competition playfully and thought that I will spend a good weekend on this problem. But the competition grew complex from a simple enough problem and ended up me being tackled by the problem; instead of me tackling the problem. This article will summarize my mistakes and the 2 specific things I learned from this. Summary of the article: Recently I took part in a hackathon, and tried my ml expertise with a tabular data. But for some reasons, I wasn't able to tackle the problem. The main findings were I used a wrong encoding method, as well as didn't do enough model testing. The good: In this project, the problem was to predict for a healthcare company whether they will get a customer or not; based on different policy based data. The dataset consisted of different data about the customer and policy amount etc. The customer related data were maximum and minimum ages mentioned in the policy, for how many years the policy have been work...

 Introduction: I have recently started learning react in a linux machine and as I proceeded, I faced a number of issues with react,npm and npx. As I tried searching in net, it seemed that many of these are not well explained or documented together. In this post, we are going to document the errors and the solution which worked for me. Error 1: npx create-react-app fails to run with error. For this, I found this stackoverflow question to be helpful. If your npx fails, chances are that you have to first start your npm and then you need to run the npx command. The relevant code to run is: npm init npm install create-react-app npx create-react-app myapp Error 2: NPM — npm start Error: ENOSPC: System limit for number of file watchers reached   For this error, I followed this guide blog post . But sadly the solution is slightly incorrect. First, let's understand the problem. Max user watch error occurs in a linux machine because the number of files the system watches/monitors ...

                               What is topic modeling?                                      Written by: Shyambhu Mukherjee Introduction: Topic modeling is one of the famous natural language processing tasks. Topic modeling refers to assigning one or multiple topic to a set of document. In this article, we will discuss the basic, medium and advanced understanding of topic modeling and discuss multiple python libraries which will be used to do topic modeling. Summary of the article: In this article, first we will define what topic modeling and topic classification modeling are; what are their difference and how to do both as a overview. Then we will discuss to...