Dynamic Attribute Graphs-based Controlled Text Generation

Xun Liang^1*, Hanyu Wang^1*, Shichao Song^1*, Mengting Hu², Xunzhi Wang², Zhiyu Li^3†, Feiyu Xiong³, Bo Tang³

ACL'24 Findings
1 Renmin University of China
2 Nankai University
3 Institute for Advanced Algorithms Research (Shanghai)
^*Indicates Equal Contribution
^†Corresponding Author

Paper Code arXiv

Introduction to Controlled Text Generation

Controlled Text Generation (CTG) focuses on generating text that adheres to specific attributes like sentiment and non-toxicity, maintaining both the quality and generative power of Large Language Models (LLMs). Traditionally, smaller language models have been used to guide the decoding of larger models, offering control but often at the cost of reducing output quality and variability.

Recent research highlights a key dilemma: excessive control can undermine text fluency, turning advanced LLMs into mere "puppets" of smaller models. This excessive reliance compromises LLMs' inherent decoding capabilities, relegating these powerful models to subordinate roles.

This introduction underscores the need for a balanced approach to integrating control within LLMs, ensuring that enhancements in text generation do not sacrifice the models' intrinsic strengths.

Semantic Influence of Key Words

In light of our exploration, we believe that the specific attributes of a text are predominantly determined by a limited number of words that bear a close relation to those attributes. Although these key words are sparse within the text, their impact on the overall attributes is decisive (See Figure 1).

Figure 1: Example demonstrating the influence of key words on text attributes.

Illustration of Semantic Space Influence

Figure 2: Illustration of the impact of key words on text attributes within the semantic space.

For instance in Figure 2, changing the word "masterpiece" to "failure" in the sentence "The novel is a masterpiece of storytelling, with a complex narrative." shifts the sentiment from positive to negative. This change not only alters the entire sentence's sentiment but also its meaning. In the conceptual framework of semantic space, these attributes can be seen as dimensions within this space. By strategically adjusting these key words, we can guide the text generated by Large Language Models (LLMs) to move in the desired direction within the semantic space, controlling its attributes without significant alterations to the overall content.

Our Approach: Dynamic Attribute Graphs-Based Controlled Text Generation (DATG)

Building on our insights into controlled text generation, we introduce the Dynamic Attribute Graphs-based Controlled Text Generation (DATG) method. This refined approach leverages dynamic attribute graphs to identify and modulate a subset of key attribute words, allowing for targeted adjustments that enhance the precision of attribute control in generated text.

Figure 3: Framework of DATG Method.

DATG begins with Contextual Corpus Construction, where large language models (LLMs) generate text sequences from specifically designed prompts to capture the desired attributes. The texts are then assessed through Attribute Classifier Scoring, employing classifiers like toxicity or sentiment models to evaluate their alignment with the target attributes.

At the heart of DATG is the Dynamic Attribute Graphs Construction step, which transforms these text sequences into directed weighted graphs based on classifier scores. This involves creating positive and negative attribute graphs, which represent the adherence to and deviation from the desired attributes. These graphs guide the subsequent text generation to ensure that it remains within the desired semantic dimensions.

The process concludes with ReGeneration with Dynamic Boundary Controlling. Here, a graph ranking algorithm selects key nodes, which are critical for pushing the generated text towards the control attributes' upper boundaries in the semantic space. Adjustments to these key nodes, facilitated by logits-boost and prefix-prompt strategies, refine the text generation process, ensuring that it adheres closely to the desired attributes while maintaining semantic integrity and coherence.

Results for Experiment

Effectiveness and Fluency: The DATG approach ranks highly in both toxicity mitigation and sentiment transformation tasks, effectively reducing unwanted attributes while maintaining text fluency. This demonstrates the method's ability to produce high-quality, coherent text across different contexts and requirements.
Attribute Control Validation: The success across various datasets confirms our hypothesis that adjusting a few key attribute words can effectively control the text's overall sentiment or toxicity. This strategic modification ensures that the changes in attributes do not compromise the natural flow and coherence of the generated text.
Consistency Across Models: The DATG method shows consistent performance in reducing toxicity and transforming sentiment across different LLMs and datasets. This stability underscores the robustness of our approach, highlighting its adaptability to different LLMs without losing quality.

Speed Advantage: DATG exhibits faster generation speeds compared to PREADD and FUDGE, emphasizing the efficiency of our approach even when integrating complex attribute control mechanisms.

Computation Times for Each Stage of DATG

Figure 4: Detailed computation times for each stage of the DATG process.

Potential for Speed Improvement: Figure 4 illustrates the average computation times for each stage of the DATG process, using Alpaca-7B as a case study. Notably, the stages of Contextual Corpus Construction and Dynamic Attribute Graphs Construction are highlighted as primary areas for achieving significant time reductions. By pre-generating extensive attribute graphs, we can allow for quicker identification and integration of relevant sub-graphs and nodes during the generation process. This pre-construction strategy significantly reduces the computational load during these stages, thereby enhancing the overall speed of generation and enabling faster response times in practice.