Once you start looking at a problem, there are a wide variety things you want to try. Everything from related fields, to technical details might call for new analysis! Here is a public listing.
Technical details: character level vs word level, the specific architecture, and whether code exists
- Style Transfer High and Low: Character-level and Multi-dataset Thematic Bonus
This research project looks at performing style transfer using character level embeddings/language model, and then also using multiple datasets. In this way, we can finally solve the problem of style transfer: from the ground level, and also from a higher-sentence level.
Exploring style transfer via text summarization We can create the bridge between style transfer and text summarization; perhaps leveraging multi datasets. For instance, we can combine the style transfer task with the text summarization task, and see what that produces.
Style transfer with multiple datasets What exactly is going on here? When we have multiple datasets, what happens!
Style transfer with exotic architectures Transformers, character-level transformers, seq2seq, etc.
One interesting idea I want to quickly investigate is how we can do style transfer with text summarization; treating that as a dataset that we use.
This is ideal since it uses pytorch and also the pointer networks!
Some architecture details: For the final network, we can implement it via training 3 separate decoders, which backprop on the same encoder. Or we can add 3 heads to the decoder, and train each separately. It becomes challenging to say what the head is, when we have an RNN. in this case, the output should be different size(!).
Text is way harder than images. Just in the loss/classification stage, there is SO much to be explained in text, as opposed to images, where we just have a single step.
We probably need a multi-task training reference too; regarding the order in which we train things! We need to examine some literature to get this paper
Hence, we can sketch the current architecture
We want the config vocab size to be all 3
We can break into research in the following ways: