Move/Mirror/Migrate all your bitbucket repos to github

1.Add ssh keys to your bitbucket and github accounts

https://confluence.atlassian.com/bitbucket/set-up-an-ssh-key-728138079.html#SetupanSSHkey-ssh2

https://help.github.com/articles/adding-a-new-ssh-key-to-your-github-account/

2.Generate github token for reading repo info and read/write permission for creating repos (you can give all permission except delete )

https://github.com/blog/1509-personal-api-tokens

3.Use this script ‘bb2gh.py’ @ https://github.com/nmi/bb2gh or (https://github.com/kanihal/bb2gh)to mirror the repos from bitbucket to github.

 

Advertisements

Quotes

all models are wrong but some are useful – https://en.wikipedia.org/wiki/All_models_are_wrong

archive.org tools and stuff

http://machawk1.github.io/wail/

upload a site to archive.org

http://archiveteam.org/index.php?title=Internet_Archive#Uploading_to_archive.org

1.crawl save each webpage seperately

 wget http://web.archive.org/save/

2.https://github.com/n0tan3rd/wail

make warc file and upload to archive.org

http://archiveteam.org/index.php?title=Frequently_Asked_Questions

I uploaded a WARC file but why doesn’t it show up in Wayback Machine?

To ensure content integrity, items with WARC files must have the mediatype set to “web” and be under the Archive Team collection in order for it to be ingested by the Wayback Machine.

 

locally browsing with warc file –

https://github.com/webrecorder/webrecorderplayer-electron

 

Tools 

https://webrecorder.io/

http://rhizome.org/software/

https://ws-dl.cs.odu.edu/Main/Software

http://archive.org/help/abouts3.txt – s3 api key -> http://archive.org/account/s3.php

 

download a site/page from archive.org for given time

modify and use this script

 

https://gareth.halfacree.co.uk/2013/04/bulk-downloading-collections-from-archive-org

 

Drive sync options on ubuntu

1. insync is the best option that is available for ubuntu (if you are willing to pay for it)

2. odrive (you can mount only one account at a time)

Download and install

https://docs.odrive.com/docs/odrive-sync-agent

od="$HOME/.odrive-agent/bin" && curl -L "http://dl.odrive.com/odrive-py" --create-dirs -o "$od/odrive.py" && curl -L "http://dl.odrive.com/odriveagent-lnx-64" | tar -xvzf- -C "$od/" && curl -L "http://dl.odrive.com/odrivecli-lnx-64" | tar -xvzf- -C "$od/"

Add theses to end of your .bashrc file

export PATH="/home/jkl/.odrive-agent/bin:$PATH"

alias startodrive= "/usr/bin/nohup /home/jkl/.odrive-agent/bin/odriveagent > /dev/null 2>&1 &"

Start the background service

startodrive

Goto odrive site -> settings -> get your token

odrive authenticate your_auth_token

Link your local folder with corresponding remote drive/folder

 odrive mount /media/jkl/64BCC6E1BCC6ACBC/Users/jkl/Google\ Drive/ /jagadkanihal

 

backup (send from local to remote)

odrive backup /media/jkl/64BCC6E1BCC6ACBC/Users/jkl/Google\ Drive/ /jagadkanihal

 

sync (remote to local)

Extension meanings

  • .cloudf  : cloud folder
  • .cloud : cloud file
odrive sync jagadkanihal.cloudf

 

3. Duplicati (compressed/encrypted one way backup : local to remote)

add backup -> step2. destination -> click on the authid (opens drive – login – fetches auth id) -> path on server should folder name (if left blank, it will choose the main/login folder )

Duplicati

clipboard Append

Appending content to clipboard without overwriting
Here is the script which essentially does this.

clip_append_n.sh

#!/bin/bash
#text under selction i.e. highlighted
p=$(xclip -selection primary -o)

# text already in the clipboard , (what happens if there is image in clipboard)
c=$(xclip -selection clipboard -o)

#seperator
sep=$'\n'
echo "$c$sep$p" | xclip -selection clip
notify-send --expire-time=1000 "ClipAppend" "Appended with newline"

#ubuntu ignore that expire time option hence - workaround hack
sleep 1
pkill notify-osd

Add keyshortcut to run this script:
System settings>> Keyboard and click on Shortcuts tab. Click on Custom Shortcuts and then click on + sign to define your key combination. In the command box put:

clip

bash -c 'bash ~/clip_append_n.sh '

click on disabled on the right side of the entry , you’ll get new accelerator visible then put your desired “KEYS” for the shortcut

I use ALT+N to trigger the action appending highlighted content to clipboard with a newline as a seperator.

References:

http://askubuntu.com/questions/797132/how-to-only-append-to-clipboard-but-not-overwrite-current-data

http://askubuntu.com/questions/873948/automatically-modify-clipboard-content-regex-pattern-replacement-when-copy-pas?noredirect=1&lq=1

http://stackoverflow.com/questions/42507916/how-can-access-read-and-write-system-clipboard-from-a-kernel-module

Set up qtcreator for kernel module developement (kernel programming)

linuxpenguinqt

Why use qtcreator?
qtcreator is one of the best ide for c/c++ developement. It has amazing autocompletion features. It makes code walktrough of large projects (like linux kernel code) easier.

1.Import existing makefile project into Qt Creator
If you are using your own build system, you create a project by selecting “Import of Makefile-based project”. This will create some files in your project directory including a file named .includes. In that file, simply list the paths you want to include, one per line. Adding “include paths” to qtcreator for kernel projects  enables autocompletiton for kernel related sources (module.h ) on Qt creator.:

All this really does is – tell Qt Creator where to look for files to index for auto completion. Your own build system will have to handle the include paths in its own way.

for eg. for project ‘dummy’ , the include file would be ‘dummy.includes’ which should contain path to kernel source

/usr/src/linux-headers-4.4.0-92-generic/include/

Reference:
http://stackoverflow.com/questions/2752352/how-to-add-include-path-in-qt-creator

 

2. Makefile and build(compile)

Makefile (for module file jaggi-klm.c file)

obj-m += jaggi-klm.o 

all: 

        make -C /lib/modules/$(shell uname -r)/build M=$(PWD)/dumkp modules

clean: 

          make -C /lib/modules/$(shell uname -r)/build M=$(PWD)/dumkp clean

builds module file with extension .ko (kernel object)

 

3. Then do load/unload the module using:

sudo insmod  _____.ko
sudo rmmod   _____.ko

 

 

References:

iitb email on gmail/thunderbird

On Gmail App

CSE mail

https://www.cse.iitb.ac.in/~swlab/Thunderbird_CSE.pdf

There is a typo in smtp part, instead of imap it should be smtp

GPO mail

http://www.cc.iitb.ac.in/engfaqgeneinf/10-english/articles/28-information-on-setting-up-imap-based-email-clients

outgoing smtp starttls(all certificates accepted)

__________________________________________________________________________

On thunderbird

CSE mail (same as above)

https://www.cse.iitb.ac.in/~swlab/Thunderbird_CSE.pdf

There is a typo in smtp part, instead of imap it should be smtp

GPO mail

http://www.cc.iitb.ac.in/engfaqgeneinf/10-english/articles/28-information-on-setting-up-imap-based-email-clients

incoming imap – ssl/tls port 143- plain password

 

____________________________________________________________________________

— gmail on thunderbird

https://productforums.google.com/forum/#!topic/gmail/3jO9r667zRg

https://askubuntu.com/questions/910614/why-cant-i-progress-beyond-entering-my-username-when-configuring-thunderbird-to/910630#910630

most probable cause iitb proxy/firewall not allowing mail ports to google

How to get most out of ML course (CS-725) @ iitb?

This blog post is meant for students at IIT Bombay considering to take Foundations of Machine Learning (CS-725) Course taught by prof. Ganesh Ramakrishnan.

Prerequisite: To do well in this course, you should be thorough with Linear Algebra (LA) and Probability Theory(PT) concepts.

If you are already familiar with LA and PT, you can skip this part.

For Linear Algebra (LA), I suggest you go through “Linear Algebra and Its Applications” Book by Gilbert Strang along with his MIT OCW videos  , Once you do this, you should be familiar with things like rank, row space, column space, null space of a matrix, least squares method(*), eigen values, and various decompositions of matrices like Eigendecomposition, SVD, QR, LU etc.

images  images-1

For Probability theory, go through “First Course in Probability” book by Sheldon Ross along with MIT OCW videos of prof. Tsitsiklis. Once you go through this, You should be thorough with Bayes rule(prior, posterior, likelihood), distributions like Gaussian(* very imp), binomial, beta etc

415tc8ue1l-_sx398_bo1204203200_  hqdefault-1

Extra prerequisite for non cs students, As this course involves 2 programming assignments and 1 project, you are expected that you should be able to use atleast one of  python/R/matlab languages effectively.

Now Coming to actual course content,

For 2016b(Autumn), the course content was properly organised compared to the previous offering(2016a Spring) of the same course. Before midsem, supervised regression models were covered  and post midsem supervised classification models and a little bit of unsupervised clustering models were covered.

2016a(Spring) course content is available here

2016b(Autumn) course content is available here

The main reading material for the course are class slides and tutorials that are given in the class. There are short lecture videos of class lectures that are available on course site on bodhi tree. Even if you miss the lecture, I would suggest you not to miss tutorial solutions discussion.

Whatever I mention below are for understanding and auxillary.

I personally preferred using “Pattern Recognition and Machine Learning” Book by Christopher Bishop as reference book. Along with the book, I used AI course videos of patrick H winston available @ mit ocw. These videos were particularly helpful for support vector machines(SVM), decision trees and neural networks. He is an amazing orator, conveys the content in simple terms without losing out on mathematical rigour. If you get time, I would suggest you to watch all his videos.

images-2

Before midsem:

For Linear, Ridge, Lasso, I studied mainly from online resources.

These are some of the material on youtube that I found useful (in no particular order).

mathmonk

Ritvik

patrickJMT

Alexander Ihler

Alex Smola

For support vector regression(SVR) – I think winston’s lecture video on svm + it’s mega recitation video would be sufficient

For KKT optimization conditions, This link  would be useful.

mercer kernels (I still haven’t completely understood these things, I’ll update once I find any good resource)

Post Midsem:

For perceptron, sigmoidal classifier (aka logistic regression):  class slides and tutorials should be sufficient

Here is a simple proof of convergence of perceptron that I found online.

For Neural Networks(*), CNN, RNN, LSTM: I started with  Patrick H winston’s AI video on neural which gives the background and overall idea of a neural network and back propagation which is a must for understanding the working of neural networks.

Once you go through the Patrick’s video, you can watch Andrej Karpathy’s CS231(stanford) videos, this guy and his blog is amazing . I actually binge watched all videos in that playlist over 2 days, really enjoyed it.

For Decision Trees – Adaboost, Support Vector Classification : Again Patrick’s videos would suffice along with slides.

For unsupervised learning: (again haven’t quite understood this one also, will update this one once I find any good resource for this)

I just found this Presentation by Yann LeCun contains overview of lot many topics ML, DeepLearning (differences + overlap -> slide 24) http://www.cs.nyu.edu/~yann/talks/lecun-ranzato-icml2013.pdf 

LAMP and XDEBUG setup on ubuntu

Use tasksel to install LAMP

16.04 – https://askubuntu.com/a/914181/650264

sudo apt install tasksel
tasksel 

and xdebug

sudo apt install php-xdebug

xdebug configuration on the server

cd /etc/php/7.0/fpm/conf.d/
#/etc/php/7.0/mods-available/xdebug.ini
sudo vim 20-xdebug.ini
zend_extension=/usr/lib/php/20151012/xdebug.so
xdebug.idekey=PHPSTORM
xdebug.remote_enable = 1

;one of these should be set
;set this where ide is running(check ifconfig and put below) and for any client - android etc
;xdebug.remote_host = 192.168.0.101 (won't work, should be a unique ip in the network of server,wifi temp ips wont work)
xdebug.remote_host = 10.1.224.101

;set this for ide running anywhere (roaming laptop) and client calling must also be the machine\
;(because xdebug uses request to determine the ip of ide to send debug info)
;xdebug.remote_connect_back = 1

xdebug.remote_port = 9000

;if set, debugs without that cookie requirement
xdebug.remote_autostart=1

xdebug.max_nesting_level = 400
xdebug.scream = 0
xdebug.cli_color = 1
xdebug.show_local_vars = 1

sudo service php7.0-fpm restart 
sudo service apache2 restart

xdebug setup for phpstorm ide as debug client

 

 

+Another Video

 

IIT Kanpur MTech Selection Process (7th May 2016)

Roughly 300 candidates were called for M.Tech written test which was held from 9am to 11am followed by a programming test of 90 mins duration held from 11.30 am to about 2 pm (including instructions on how to use web editor/evaluation framework that was provided). After both of these test,@ around 4pm, 120 candidates were shortlisted for subjective test which was held from 5pm to 7pm.

Written test consisted of objective, fill in the blank, short answer type questions divided into 3 groups. Candidates were supposed to answer questions from any 2 groups. If you answer questions from all 3 groups, best of 2 group marks are considered for short listing.Candidates were not allowed to take the question papers with them, so whatever I can remember from my memory, I have jotted down below.

A.Written Questions : (3 groups, 30 questions)

1.Theory

  1. Given complete graph Kn, what is number of distinct spanning trees possible from DFS? BFS?
  2. Given adjucency matrix A of a graph, what is expression/algo for Reachability matrix?  time complexity for this computation?
  3. 2202 (mod 101) =_____
  4. T(n)=t(√n)+O(1), closed form expression for T(n)=____
  5. S→aS|B, B→bB|ε, the language generated by the given grammar  is _____ (multiple choices were given)
  6. regEx which does not generate 2 consecutive b’s  are ____ (multiple choices were given)

2.Systems

  1. Linux command for searching all occurences of “hello” in file.c ______
  2. Which segment in the process memory structure are the addresses of Dynamic Link Libraries (DLL) stored?___
  3. (pipeline related question) Given Assembly Language instructions, calculate clock cycle difference with full bypass pipeline and non bypass pipeline?____
  4. (another pipeline related question) similar to above question And (2 questions on cache memory and memory management )

3.Statastics and DataScience

  1. (conditional probability related question) 10% of sick kids have measles and other 90% have flu. 80% of sick kids having measles develop skin rash where as 1% of sick kids having flue develop skin rash. what probability of a sick kid who is found to have rash is suffering from measles?
  2. Batman and Robin are trying to keep the city of Gotham crime free.The criminals however come up with a plan to commit the crimes according to following schedule.
    10-11 criminal A
    11-12 criminal B
    12-1  LUNCH BREAK (no crimes committed during this time)
    1-2 criminal BEN
    2-3 criminal C
    3-4 criminal D
    Batman or Robin alone can defeat each of the criminals except for criminal BEN. Both batman and robin should fight together to defeat criminal BEN and each alone can’t defeat him.
    Batman and robin each independently select one of the above time slots and go fight crime.
    a. what is probability that exactly one criminal is defeated in a given day?
    b. what is probability that exactly 2 criminals are defeated in a given day?
    c. what is probability that exactly no criminal is defeated in a given day?
    d. what is probability that exactly at least 1 criminal is defeated in a given day?
  3. Each point in a x-y plane can be colored with red or blue. what is the min number of points however colored should be able get separated by a line such that all red colored points will be on one side and all blue colored points will be on the other side?
  4. given joint probability table, fill in the missing blank type question(something like this)
    joint probability table
  5. Which of the following is not a valid probability density : -0.01,  1,  100, 1.1
  6. Given communication channel with X as input and y as out. X can take 0 or 1 and are equally probable. channel matrix is given below. Find the entropy.(I don’t remember source entropy or conditional entropy)
    communication channel

 

B.Programming test : (3 problems to be implemented in C language)

  1. Program for Counting Number of Inversions in a sequence numbers
  2. Given 2D array, program to find sum of elements a[i][j] which satisfy sum of elements of ith row = sum of elements on jth column
  3. program to find Levenshtein distance between 2 strings. Refer Levenshtein distance (this is kind of complementary problem to Longest Common Subsequence (LCS) DP algo).Although the word “Levenshtein distance”  was not mentioned,  the described problem in essence is the same.

 

C.Subjective test  : (again 3 groups,  about 10 questions in total)

1.Theory

  1. prove  that If a graph is not (n-1) colorable, then the graph is a clique.
  2. prove or disprove, If a graph is  having independent set of size k, then can be colored with (n-k+1) colors
  3. prove or disprove, If a graph having (n-k+1) chromatic number, then it has independent set of size k (it is actually the converse of the previous question statement)
  4. write an algorithm to find a cycle in a graph in O(n) time. prove that the time it takes is O(n).

2.Systems

  1. 1 reason for using multilevel page table
  2. find the problem in the following implementation of removing an element from que1 and adding it to que2. initial value of binary semaphore is 1.
    Q1→wait();
    Q2→wait();
    E→Q1.deque();
    Q2→signal();
    Q1→signal();
  3. write the correct implementation for the above operation.
  4. given 4 level page table, 4KB page size, 8B Page Table Entry size, what is virtual address size/space?

3.Statastics and DataScience

  1. X and Y are independent random variables, give expression for E[X], E[Y] and prove that E[XY]=E[X] E[Y]
  2. u and v are eigen vectors of real symmetric matrix, prove that uvT=0
  3. (one question on convex analysis)
  4. (one more question from linear algebra)