Google Summer of Code – Coding Phase1

by

One of the requirements of my organisation & GSoC is to keep weekly blogs. This will help your mentors to understand your progress and make a decision during the Phase 1 evaluation about your progress. It is the progress and the effort that matters more than the actual outcome. You may not understand the details of the work here, but going through these weekly blogs, you can get an overview about how it should be for your project. If you want to know more about the project, you can visit here.

Week 4 (June 13 – June 18)

Ambitions

  • Creation of Audio Files from Redhen Video files
  • Run Yamnet on one RedHen video to extract the classes labelled for the audio effects
  • Execute the baseline code in CWR HPC servers

Challenges

  • Redhen does not have an existing pipeline to extract the audio. So for now, I am writting a script to do that myself. (This has been done locally). A generic pipeline to reuse for all RedHen projects can be done (but out of scope for my work). This can be taken after the work is completed.
  • Deciding output format is one challenge. Currently I am dumping the data (scores & labels) in the CSV format.

Strategies

  • Handling the tasks & milestones through Github project is going on good for now. All issues and discussions are being handled there.
  • All the discussions with the mentors are also handled in the Github project.
  • Prof Urhig also suggested to write the output in the format of ELAN EAF files. Prof Turner suggested to create a file with .sgx extension and generate data in Red Hen format. These options should to be weighed in the M2 milestone after the integration with the Redhen infrastructure.

Setbacks

  • All this work has been done locally. I need to set up the docker and singularity container to make this happen. This will be taken in the next week.
  • All the work for generation of output files is also not completed. It is important to decide how to parse the output.

Details/Successes

  • Set up the Github Project (as suggested by mentor Austin)
  • Was able to write a script to extract the audio files from the videos
  • Dumped the audio tags in a csv format
  • Discussion was done with mentors in slack/github project

Week 5 (June 19 – June 25)

Ambitions

  1. Study about Singularity and Docker environment
  2. Create Github workflows and modify docker file to download the docker hub in local environment
  3. Prepare singularity containers in local environment.
  4. Generate audio files from video files using the singularity

Challenges

In the first step, I was trying to set up the docker image in the CWR HPC cluster itself. But there was a problem in submitting the SLURM jobs from gallinahome. So Prof Uhrig suggested to create the Singularity containers on our personal Linux machines (physical or virtual).

So to create the singularity container, we first have to create a docker image. The idea is to set up a docker image locally and push it in docker

Details

In this week I tried to set up the docker container for setting up the project infrastructure.

Installation & Account Creation

I installed singularity to use it later. I will try to create the singularity container in local and import it in the HPC itself.

Singularity

I followed the installation process as mentioned in the Singularity Website.

I first installed the dependencies. sudo apt-get update && sudo apt-get install -y \ build-essential \ uuid-dev \ libgpgme-dev \ squashfs-tools \ libseccomp-dev \ wget \ pkg-config \ git \ cryptsetup-bin

Then I installed Go version 1.16.6.

export VERSION=1.16.6 OS=linux ARCH=amd64  
wget -O /tmp/go${VERSION}.${OS}-${ARCH}.tar.gz https://dl.google.com/go/go${VERSION}.${OS}-${ARCH}.tar.gz && \
sudo tar -C /usr/local -xzf /tmp/go${VERSION}.${OS}-${ARCH}.tar.gz
echo 'export GOPATH=${HOME}/go' >> ~/.bashrc && \
echo 'export PATH=/usr/local/go/bin:${PATH}:${GOPATH}/bin' >> ~/.bashrc && \
source ~/.bashrc

Then I downloaded and installed Singularity from a Github release

export VERSION=3.8.1 && wget https://github.com/sylabs/singularity/releases/download/v${VERSION}/singularity-ce-${VERSION}.tar.gz && \
tar -xzf singularity-ce-${VERSION}.tar.gz && \
cd singularity

./mconfig && \
make -C ./builddir && \
sudo make -C ./builddir install

To test that the singularity is working fine. I built a container in the Sylabs Container Library.

sudo /usr/local/singularity/3.8.1/bin/singularity build --sandbox niftynet niftynet.img

Docker & Docker Compose

I installed the docker version 20.10.12. I faced some issues while installing the latest version of docker-compose. This stack overflow link helped me out. I used the following command

sudo curl -L "https://github.com/docker/compose/releases/download/$(curl https://github.com/docker/compose/releases | grep -m1 '<a href="/docker/compose/releases/download/' | grep -o 'v[0-9:].[0-9].[0-9]')/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

sudo chmod +x /usr/local/bin/docker-compose

Setting up github workflow and docker

I fixed the Github workflow and Dockerfile in my repo such that I can create the docker using the workflow. I also wrote a docker-compose file.

Setting up Local Docker Image

Once the workflows are set up, I cloned the repo and try to build the docker using the command docker-compose up  Now the docker image will be build locally.

Once this is done, we can check the docker image using command “sudo docker images"

Testing the docker image

To test the created docker image, we can give the following command 

sudo docker run -it --rm -p 8888:8888 -v $PWD:/TaggingAudioEffects technosaby/gsoc2022-redhen-audio-tagging

As I did not have GPU in my local system, so I did not use (–gpus all) flags

Build a local singularity sandbox/image from docker container

Either we can build a sandbox from the docker container, sudo /usr/local/bin/singularity build --sandbox singularity_container technosaby/gsoc2022-redhen-audio-tagging or, we can build an (.sif) image, sudo /usr/local/bin/singularity build image_audio_tagging.img docker://technosaby/gsoc2022-redhen-audio-tagging

Copy the image to the HPC

We can do this by SCP, scp image_audio_tagging.img sxg1263@rider.case.edu:/home/sxg1263/sxg1263gallinahomeWe can also copy the sandbox using this command, scp -r singularity_container/ sxg1263@rider.case.edu:/home/sxg1263/sxg1263gallinahomebut the sandbox is larger is size than the image, so it is better to use the .sif image.

Running the container in HPC

After loading the singularity using the command, module load singularity/3.8.1 we can run the container using,

singularity shell -w /home/sxg1263/sxg1263gallinahome/singularity_container

Setbacks

  1. I have done the set up of docker but I could not finish the work for running my script for conversion of the audio files
  2. I also spent a lot of time learning about docker and singularity which impacted my ambitions.
  3. I also need to convert the docker image in SIF in HPC which will be taken care in the next week.

Strategies

I created a free docker hub and created repository with name “gsoc2022-redhen-audio-tagging” on my username “technosaby”. After creating the local docker container, we could also push it to docker hub The command for that is docker push technosaby/gsoc2022-redhen-audio-tagging This will push all the local docker changes in the docker hub.

After that we can clone the docker hub image in the HPC As the docker image is ready in the dockerhub, we can pull the image in the HPC using the following command.

docker pull technosaby/gsoc2022-redhen-audio-tagging

Then we can convert Docker image to SIFs and also run in HPC.

But this has some challenges. Docker hub has restrictions on free license of size of the docker. So, I will be following the approach of creating the container locally and copying to HPC

Tips

SLURM will by default write logs to the directory from which the job was submitted. This is the current working directory from where I am running “srun” or “sbatch”. So we have to make sure that we always run “srun” or “sbatch” while in home directory or a subdirectory thereof. Something like ~/logs (in the home directory) can be good place to run it from.

Week 6 (June 26 – July 2)

Ambitions

  1. Do a baslining with YaMNet
  2. Generate a RedHen output metadata file

Challenges

Finalising the metatdata format is a critical part because these metadata is the only way how the future RedHen’s will use the output

Details & Successes

In this week, I worked on the following

  1. Created a pipeline with the YaMNet model and run some RedHen Videos.
  2. Created a metatdata file (.sfx) format based on the output of the model. The generated template contains the following
    • Header: Red Block, Copied from seg files of the video
    • Legend: Blue Block, Describing about the owner of this file
    • Body: Green Block showing one frame, where all the audio tags are mapped corresponding to time frame 

Also we had a discussion and review about the progress. Prof Turner suggested the next set of events

  • Create a bash script to filter the tags based on timeframes
  • Use the generated Metadata in ELAN
  • Prepare a codebook explanation about the structure of the generated output

Leave a Comment