Affine and Perspective Transformation

In affine transformation (link, link2), all parallel lines in the original image will still be parallel in the output image. To find the transformation matrix, we need 3 points from input image and their corresponding locations in output image. Then cv2.getAffineTransform will create a 2×3 matrix which is to be passed to cv2.warpAffine. Affine transform can perform rotation, translation, resizing,

pts1 = np.float32([[50,50],[200,50],[50,200]])
pts2 = np.float32([[10,100],[200,50],[100,250]])

M = cv2.getAffineTransform(pts1,pts2)
dst = cv2.warpAffine(img,M,(cols,rows))

For perspective transformation (see links above), you need a 3×3 transformation matrix. Straight lines will remain straight even after the transformation. To find this transformation matrix, you need 4 points on the input image and corresponding points on the output image. Among these 4 points, 3 of them should not be collinear. Then transformation matrix can be found by the function cv2.getPerspectiveTransform. Then apply cv2.warpPerspective with this 3×3 transformation matrix.

pts1 = np.float32([[56,65],[368,52],[28,387],[389,390]])
pts2 = np.float32([[0,0],[300,0],[0,300],[300,300]])

M = cv2.getPerspectiveTransform(pts1,pts2)
dst = cv2.warpPerspective(img,M,(300,300))

In summary,

  • Affine transformation preserves lines and parallelism.
  • Perspective transformation preserves lines. Affine transform is a special case of perspective transformation.
  • PS, affine transformation does not preserve angle. Conformal transformation preserves angle.


Affine and Perspective Transformation

Why openCV uses BGR (not RGB)

When using openCV to load an image, the color information is stored in the order of Blue-Green-Red instead of the currently more popular scheme RGB.

import cv2
image = cv2.imread(args["image"])
cv2.imshow("Image" , image)

This reads in and displays the correct image file. An alternative way to do this using matplotlib is as follows.

import matplotlib.image as mpimg
image = mpimg.imread(args["image"])
# plt.imshow(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))

However, if we read in the image file through cv2 and display it with matplotlib or vice versa, the image will not be displayed correctly, since the R and B channels are flipped (see above link for an example image). Luckily, cv2 has a built-in way to correct this.

import cv2
import matplotlib.image as mpimg
image = cv.imread(args["image"])
plt.imshow(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))

Alternatively, we can hack this by swapping the B and R channel since it is the third dimension of the image.

image = image[:, :, ::-1] # or image = image[:, :, (2, 1, 0)]

According to the following post, BGR was introduced to the openCV in a time when BGR was the most popular format, and it got stuck. It is very similar to the funny story why US railway gauge is 4’8.5″.


Why openCV uses BGR (not RGB)

Installing OpenCV3 and Python3 in virtualenv on MacOS Sierra

I spent almost 3 hours today and finally got CV3 installed. I found this detailed tutorial by Andrian Rosebrock early on:

My Hackintosh desktop has not been upgraded for 3 years after it was built in 2013 (running Mountain Lion), and there was no way to get homebrew properly installed due to “Command Line Tool version too old” error. So I decided to upgrade my MacBook Air to the latest MacOS Sierra and install OpenCV on it, also to get a more (physically) portable development environment.

One thing I had installed on my MBA is Anaconda. This, as it proved later on, made my installation a bit more complicated and I had to make adjustment to the tutorial to make OpenCV work.

$ python --version
Python 2.7.10 :: Anaconda 2.3.0 (x86_64)
$ conda --version
conda 3.14.1

Here is a list of adjustments I had to make:

  • is not in /usr/local/bin/, but rather in

    /anaconda/bin/. Specify this path instead in ~/.bash_profile. 

  • After installing virtualenv, when creating virtualenv named cv and later calling workon cv, I got the following error message:
    Usage: source deactivate
    removes the 'bin' directory of the environment activated with 'source
    activate' from PATH. 

The error message is a bit cryptic, but it turns out that anaconda has a command called deactivate in Anaconda’s bin. So I had to rename the activate and deactivate commands.

$ cd /anaconda/bin/
$ mv activate activate_ana 
$ mv deadectivate deactivate_ana

  • OpenCV compiled correctly, but when I tried to import it, it throw an error message “libhdf5.10.dylib not loaded opencv”. The culprit is also the previously installed anaconda.
$ sudo find / -name libhdf5.10.dylib

Luckily I found a blog post in Japanese (日本語ができてよかった!) and the blogger had almost exactly the same problem as this. The problem is solved by

$ brew tap homebrew/science
$ brew install hdf5
$ brew search

(cv) [ llgc @ megatron ~ ]$ python
Python 3.6.0 (default, Dec 24 2016, 08:01:42) 
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> cv2.__version__

Now I got Python 3.6 + OpenCV 3.2 successfully installed and imported. I will update this post if there are more hiccups caused by Anaconda down the road.

Edit: 01/11/2017

An alternative way to load and display image to cv2 is using matplotlib module. However, the latest version (matplotlib-1.5.3) does not work with virtual environment (e.g., non-system version of Python), and throws a “Framework” error The fix is to install an older version that is compatible with virtual environment (such as 1.4.3).

$ pip uninstall matplotlib
$ pip install matplotlib==1.4.3


Installing OpenCV3 and Python3 in virtualenv on MacOS Sierra

Embarking on a new journey

Our company, the MI branch of PE, has signed a deal to be bought by company V. The news came during the Christmas shutdown of our SC facility and it was a surprise for me although most of my colleagues (myself included) sensed that the management was having some secretive M&A plan going on. We met the new leadership team from our soon-to-be-mother-company this Friday and, to be honest, I was quite impressed by the eloquence and zealousness in the new CEO’s presentation on his blueprint of the merged company. However the future company he painted, a cost leader in the MI and NDT imaging component market, was hardly the technology-focused, innovative new player as I had anticipated. He even briskly denied the possibility of us entering the system integration market. That was the time when I know that I may have to reconsider my future with the current company.

I majored in physics in college and in graduate school and I always told myself that I was lucky enough to find a position closely related to my research in graduate school. And I really was, given the situation many of my classmates are in today. It also feels good to see the awe in my peers’ eyes when I inform them my job in this part of the country where almost everyone works as a software engineer — very similar experience to what I had when I informed my better half’s colleague in the financial industry that I had studied physics for the last decade of my life. I even loved my job for many months when I was learning new things about our products on a daily basis.

Now two years into my first job out of the ivory tower, my enthusiasm in the daily routine is gradually waning and I often find myself wondering about the opportunity cost (both financial and career) of me staying at the company for the next few years. Most of the time, I spend hours implementing a simple solution I came up in seconds. The job requires more attention to detail than problem solving. I believe it is time to steer my career into a slightly different course. The foreseeable future belongs to software and artificial intelligence. Working as a physicist in a hardware company manufacturing a device based on a century old concept and a decade old technology is not quite the optimal way of spending my most creative and productive years. Not to mention the new company we are evolving into is determined to wage a price war in the ever-crowded market.

Don’t get me wrong — the past two years at the company has been rewarding. I led a team to evaluate a new type of a key product component, wrote and presented a paper; my PR has been approved; went to a different country and lead the responsibility transfer on my worn. I wish when I look back at my first two years out of college at a later stage of my life, I would recount an overall positive tale to myself and my audience. However the achievements I am most proud of seem to be accomplished out of work, the DS projects, learning Cantonese and developing a fitness habit. This may suggest that my current job is not suitable for a lifetime pursuit.

The first step of switching jobs/careers is to evaluate the core skills I have developed. I believe it is a relatively easy question since image processing that is an unchanged theme in my past positions, and I like this theme to continue in my future career.

The second step is to figure out which future field I would like to be in. This is a question with more uncertainty and one that calls for more thinking. One of the field I have been interested for years is machine learning. I know ML is not my major in graduate school and I am starting to dabble though it, but based on my philosophy of not to make any choices that I would regret later, I believe my best next job would involve some degree of machine learning.

The third step is to try to establish the missing link. Image processing + machine learning = computer vision. The answer is fair and simple, so I need to brush up on the field of computer vision.

The M&A deal will close at Q2, so I have six month or so to make myself an expert in computer vision.

  • The goal: finding a computer vision related job around the end of Q2.
  • How to get there:
  1. First, use my time at work to develop useful applications using CV that should be genuinely useful for the company as well.
  2. Second, make use of nights and weekends to read books and learn new things. Dedicate at least 3 hours a day during weekday nights and 4 hours a day during weekends. That is 3*5+4*2=23 hours a week, and 6 months would be 23*26=598 hours. 600 hours should get me somewhere in the intermediate level. At least one hour a day should be spent on summarizing the topics I have learned during the day. Keep this blog to keep track of progress and organize my new thoughts.
  3. During the first few weeks, I would expect reading several books and following along different online course to get a general feeling of the field. Make plans and adapt them to keep track of time.

Resources to start with:

  1. OpenCV crash course ( The openCV book “Practical Python and OpenCV + Case Studies”( also looks good.
  2. A list of books on openCV on openCV’s website (













Embarking on a new journey