Data Analytics Team Intern, Tang Tun Yu
In the last two months, I spent my summer with Hanlun Artificial Intelligence Limited, an AI consulting firm based in Hong Kong. The main duties involved are performing three-dimensional spline interpolation and publishing an android app called “Hold On”.
Part One: 3D-spline interpolation
Hanlun AI is developing an interesting app for teaching people how to play Tai Chi (a traditional Chinese martial art). We have recorded the hand movements of different people playing Tai Chi and the recorded data are some kinds of splines. Due to the limitation of the existing hardware, we believe that our recorded data do not have enough amounts of data points to generate acceptable results. Before feeding these data into the deep learning model, we would like to have some more data points for each spline. Therefore, we decided to perform interpolations for these splines, in order to increase the amounts of data points.
Interpolation is a means to estimate an intermediate value of the independent variable of a function. There are different known approaches to deal with this task, including piecewise constant interpolation, linear interpolation, lagrange interpolation, polynomial interpolation and spline interpolation.
Before implementing these methods, let us have a look of the sample curve below. It is generated by the geomdl library available on https://pypi.org/project/geomdl/. We have 100 data points acquired discretely in our practical experiment and we would like to have 1000 data points after the interpolation for the deep learning model.
The essential idea of our spline interpolation process is to fit a piecewise cubic spline function that satisfy certain properties. The function S(x) will have to interpolate all 100 data points of the original spline and S(x), S’(x) and S’’(x) are continuous in the interval of x1 to x100. Using these properties and the fact that the second derivatives would be equal to zero at the end points (x1 and x100), the corresponding parameters of generating a unique cubic spline can be estimated. Making use of the Scipy library in python and the existing InterpolatedUnivariateSpline function, we can treat the x, y, z coordinates of the existing 100 data points as three separate splines and interpolate them. The interpolated results with 200 points output are as follows:
However, there are two major problems for this method:
1. There will be a gap between the start point and end point of the spline as the function does not interpolation from x100 to x1.
2. The distribution of the interpolated points is not equal or they are not equidistant which is not desirable.
To deal with these two problems, we tried to interpolate the average of spline and splines with partial data. We built another function based on the InterpolatedUnivariateSpline function. We take parts of the spline out and interpolate them separately. To solve the problem of the gap between start point and end point, we glue the tip and the tail of the spline by summing them. To solve the problem of uneven distribution of the interpolated spline, we take the average of all the separately interpolated splines at the end to form a final interpolated spline. The final result of 1000 interpolated points using this improved method is as follow
Part Two: ‘Hold On’
The second part of my internship is about publishing an android app the company had been developing called “Hold On”. The app is now live on google play store: https://play.google.com/store/apps/details?id=com.hanlun.holdon&hl=en_US. Please feel free to try it and tell us how you think about it. How to use the app? If this is your first try, grant camera permission for the app and reopen it. Hold on for few second and you can see that your camera become a creator of great art works. Your photo will be saved in the folder named “pictures” in your album. Here are two examples generated by two different modes of the app:
Unlike the usual filtered camera apps on the market now, our app is not post-processing the photos but pre-processing. The app made use of Gaussian Mixture Model which was first described by P.KadewTraKuPong and R.Bowden in their article“An improved adaptive background mixture model for real-time tracking with shadow detection”. Their proposed methods method is originally designed for detecting moving objects in videos from static cameras for background subtraction. Our app re-purpose KadewTraKuPong’s et al.’s method to detect outline of a scene when one “hold on” her/his camera. The rationale behind this re-purpose came from the fact that human make very subtle shaking when s/he try to hold on the camera firmly. The special effect in live can be like sketching the scene under your camera.
Remarks
At the end, I would like to thank Hanlun AI for all the insightful experience I had as an intern. I am grateful for being a part of the team and I have learnt some much about how abstract mathematical concepts can be applied in real life and some essential programming skills. Among all the amazing people I met, my supervisor, Y.K., helped me the most. His kindness and intelligence always impress me. I would also like to appreciate the efforts made by Mathematics Department, The University of Hong Kong for connecting Hanlun AI and me.
The company is currently working on two client requested projects, computer vision applications on Tai Chi machine and knowledge graph of an e-learning platform. To keep updated with us, please subscribe to this channel.
