Research Assistant in Stanford Geophysics
(2020)
Skills involved: Computer Vision, Deep Learning, Graphics, Python
I fine-tuned a Mask-RCNN model using Keras and TensorFlow to precisely detect and correct deflections in signal in order to recover decades of unreadable image data. To do so, I developed automated generation of analog seismogram images to produce a 6,405-image synthetic dataset in COCO format using over 10,000 hours of data.
Personal Project: Classical Music Period
Classification
Skills involved: Signal Processing, Computer Vision, Deep Learning, Python
This was a one-day project I did out of curiosity if an AI could hear (or, it turned out, see) the difference between different periods of classical music (Baroque, Classical, Romantic, 20th-Century). Using STFT techniques I learned in Summer 2019, I converted recordings of classical music into images and generated a 3200-image dataset. Using preprocessing functions and the baseline ResNet50 model from the fastai library, I was able to quickly fine-tune a classifier with over 90% val accuracy and 85% test accuracy. (Previous results in this area had 39% accuracy but on a different dataset. In the future, I will evaluate my model on that dataset for comparison to that paper.)
GANime: Generating Manga/Anime Character
Sketches From Drawings
Skills involved: Generative Models, Computer Vision, Deep Learning, Python
I collaborated with my friend Tai on this project. Using TensorFlow 2, Python, Google Colab, and AWS, we developed and compared the performance of three models (Neural Style Transfer, Pix2Pix, and CycleGAN) and in their ability to generate fully colorized drawings from sketches. On a sample size of 100, we achieved a FID of 220.5 and a SSIM index of 0.76 using Pix2Pix, the best model for this problem according to our evaluation. We produced a paper here, and a 3-minute summary video here.
Research Assistant in Stanford Geophysics
(2019)
Skills involved: Computer Vision, Signal Processing, Machine Learning, Data Analysis, Python
Last summer, I worked on earthquake signal processing and detection at Stanford Geophysics, where I used Python (ObsPy, NumPy, SciPy, Matplotlib, and cv2) to build a data pipeline to process (denoise) the data and subsequently detect extremely small earthquakes in the data that are difficult to find even with human expertise. The data was noisy and I used signal processing techniques such as filtering in order to remove unwanted noise but preserve useful signal. Then, by first converting the signal into frequency domain images, I was able to apply computer vision to detect and amplify edges in the signal which signified earthquakes. I then used machine learning (K-means clustering) to cluster patches of the signal into "earthquake" and "non-earthquake" signal, which allowed me to use the clusters to identify earthquake picks for new data.
TreeHacks 2019
Skills involved: Deep Learning, Python
I designed, implemented, and tuned a 5-layer Deep Neural Network (DNN) using NumPy and Keras to predict mood based on everyday habits. I managed to get fairly good results: most predictions were within 1 or sometimes 2 points on a 10-point scale from the ground truth on the test set. My teammates made an iOS app for the mood predictor where users could enter in their habits (sleep, amount of exercise, etc.) for a certain day. I then used CoreMLTools to ship the deep learning model to the iOS app.
Chatbot
Skills involved: Web Design, Language Processing, Data Structures, Probability, HTML, CSS, JavaScript
For fun, I designed a chatting interface from scratch using HTML, CSS, and JavaScript. Then, I created a chatbot AI using a decision tree and a knowledge base of vocabulary words which would respond to user messages. For a more dynamic experience, in the newer versions, the chatbot has simulated emotions which influence its messages. In one version, the emotions are modeled using a Markov chain. In the newest version, the simulation was modeled using a subset of the PAD model in psychology. Probability was utilized to simulate small changes in emotion, as well as some influence from the user's messages. Future versions would include a more dynamic version of emotion simulation which would place more weight on the influence of the user's sentiment.
