{% extends "template.html" %} {% block content %}

NLP Text Prediction (GPT-2 Model)

Project Introduction:

The inter-connectedness enabled by the Internet has resulted in an increasing amount of written digital content. Consequently, there has been an increase in the production of various digital writing tools. A common example would be Grammarly - a digital writing assistant that utilizes machine learning and deep neural networks to perform spell-checking and address grammatical issues.

These tools can assist authors in crafting their literature – whether the author is a human typing on their keyboard or artificial intelligence. Hence, we now have the capacity to construct machine-generated documents that range from hilariously awkward to near-indistinguishable from those written by a human being. They are based on statistical modeling algorithms that enable text modeling and prediction.

However, relatively little is understood about how these systems can be used to support human writers in a holistic fashion. By this, we mean in a sense that provides a greater awareness of how machine assistance affects the very nature of our written communication. One primary issue is the lack of knowledge concerning the corpus of documents that comprise the training, validation, and test sets upon which these text modeling and prediction systems are based. Is there an inherent bias in the sets that constricts the range of possibilities being provided? Another issue is that auto-completion suggestions are simply for the next sentence or phrase beyond the current. Is there a method by which we can provide not just a sentence but an entire paragraph of suggestions? Can we use the structural elements common to specific types of documents to provide a template by which to generate more diverse routes by which the author can explore their craft?

Addressing questions like these is important in this day and age where plagiarism is rampant and originality is uncertain. If there is an inherent bias in these text modeling and predictive systems, are they homogenizing our literary works – ranging from Tweets to novels – towards some unintended and potentially ill direction? If machines can generate entire documents indistinguishable from human authors given sufficient hyper-parameter tuning, do we still attribute this as a literary work of art and more importantly, attribute the human behind that knob tuning as a legitimate author?

These and other considerations should be at the forefront of the thoughts of the literary community. We live in a world where these intelligent interactive systems are always present in our daily lives. As we use them to communicate ideas with each other, should we not question the effect they have upon that communication? Are our words truly our words and our meanings truly our meanings? Or have they been subverted by these systems in the subtlest of ways?

{% endblock %}