Hello there @CharisWg

Its been a year since I code this, so I don't have the best recollection of how things work

But if I remember correctly:

• The whole process is using .py so the notebook is a bit obsolete
• The preprocessing process is done before training (no preprocessing while training), this is to reduce load on the machine
• You can go to src/data/ and you'll see both lib and pipeline, where I basically created a preprocessing pipeline
• Sadly I've lost access to the final english version of my proceeding (I'm indonesian so the thesis is in Indonesia), luckily I got a backup draft that you can check here, it sums up pretty much the whole system, the implementation in this repo should be roughly the same

Nearing the deadline of my thesis I'm not really organized in my programming and repo management, so I think that not all of the commit is in the main branch, feel free to check around other branches

This repo (for me) is complete since I've already graduated, so I won't add any changes in this repo, but feel free to ask me anything, I'll answer once I'm free