FNO for NLP

Current & Past Projects | | Links:

FNO for NLP

FNO better scaling can possibly used to create transformer architectures that scale better than current technology. Can language be described as a continuous process?

Questions

  1. Can language be described as a continuous process?
  2. Can we build transformer with better scaling properties by transforming text data into the Fourier Space?

Fourier Neural Operator (FNO)

Fourier Neural Operator is a recent technique described for instance in paper

Work

I am working on porting a technology developed for fluid dynamics and image processing into the text processing domain, as FNO scaling properties could be a game changer.

I view the work also as a practical investigation of the nature of Language.

I have achieved mixed results but am continuing to work on this in my free time.