Uncovering Theological and Ethical Biases in LLMs
An Integrated Hermeneutical Approach Employing Texts from the Hebrew Bible
DOI:
https://doi.org/10.7146/hn.v9i1.143407Keywords:
Artificial Intelligence, Ethical Biases in LLMs, Hermeneutical Approaches in AI, Computational Theology, Hebrew Bible, Theological Biases in LLMsAbstract
This paper explores theological and ethical biases in LLMs through a novel approach involving creative text generation tasks based on biblical texts, specifically the Ten Commandments and the Book of Jonah. Utilizing models such as GPT-4 Turbo, Claude v2, PaLM 2 Chat, Llama 2 70B, and Zephyr 7B, the study employs a combination of qualitative hermeneutical analysis and quantitative textual analysis. Findings reveal a prevalent progressive bias in these models, evident in their interpretations of foundational ethical guidelines and narrative texts. This bias aligns with contemporary socio-political and environmental concerns, especially in themes of environmental ethics, social justice, and inclusivity. In the narrative task involving the Book of Jonah, a dominant interpretive trend is observed, reflecting the models' tendency to mirror historical and prevailing interpretations. This study highlights the need for multidisciplinary research into LLMs' biases, particularly their impact on religious and ethical narrative interpretation and broader societal implications.
References
Anil, Rohan, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, et al. “PaLM 2 Technical Report.” arXiv, September 13, 2023. https://doi.org/10.48550/arXiv.2305.10403.
Anthropic. “Model Card and Evaluations for Claude Models.” Anthropic, 2023. https://www-files.anthropic.com/production/images/Model-Card-Claude-2.pdf.
Bolin, Thomas M. Freedom beyond Forgiveness: The Book of Jonah Re-Examined. Journal for the Study of the Old Testament 236. Sheffield, England: Sheffield Academic Press, 1997. https://doi.org/10.2307/3266995.
“Chatbot Arena ELO Calculation Notebook,” July 17, 2023. https://colab.research.google.com/drive/1RAWb22-PFNI-X1gPVzc927SGUdfr6nsR?usp=sharing.
Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. “BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding.” arXiv, May 24, 2019. https://doi.org/10.48550/arXiv.1810.04805.
Dhinakaran, Aparna. “Survey: Massive Retooling Around Large Language Models Underway.” Forbes, April 26, 2023. https://www.forbes.com/sites/aparnadhinakaran/2023/04/26/survey-massive-retooling-around-large-language-models-underway/.
Elrod, A.G. “‘Is It Right for You to Be Angry?’ Mercy, Justice, and Silence at Odds in Jonah 4:5-11.” Master’s Thesis, Southern Methodist University, 2015.
Elrod, A.G. “Nothing New Under the Sun? The Study of Biblical Hebrew in the Era of Generative Pre-Trained AI.” Hiphil Novum 8, no. 2 (2023): 1–32.
“Generative AI Could Raise Global GDP by 7%,” April 5, 2023. https://www.goldmansachs.com/intelligence/pages/generative-ai-could-raise-global-gdp-by-7-percent.html.
History of the World, Part I. 20th Century Studios, 1981.
Hutto, C., and Eric Gilbert. “VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text.” Proceedings of the International AAAI Conference on Web and Social Media 8, no. 1 (May 16, 2014): 216–25. https://doi.org/10.1609/icwsm.v8i1.14550.
Kosinski, Michal. “Theory of Mind Might Have Spontaneously Emerged in Large Language Mod-els,” 2023. https://doi.org/10.48550/ARXIV.2302.02083.
Li, Cheng, Jindong Wang, Yixuan Zhang, Kaijie Zhu, Wenxin Hou, Jianxun Lian, Fang Luo, Qiang Yang, and Xing Xie. “Large Language Models Understand and Can Be Enhanced by Emotion-al Stimuli.” arXiv, November 12, 2023. https://doi.org/10.48550/arXiv.2307.11760.
“Matplotlib.” Accessed November 28, 2023. https://matplotlib.org/.
McFague, Sallie. The Body of God: An Ecological Theology. Minneapolis: Fortress Press, 1993. https://doi.org/0.1177/002096439404800330.
Merali, Shabir, and Merali, Ali. “The Generative AI Revolution: Opportunities, Shock, and Risks.” Onward, May 3, 2023.
National Park Service. “Oregon Trail Ruts State Historic Site.” Accessed December 9, 2023. https://www.nps.gov/oreg/planyourvisit/oregon-trail-ruts-historic-site.htm.
NLTK. “NLTK Documentation,” 2023. https://www.nltk.org/.
OpenAI. “GPT-4 Technical Report.” arXiv, March 27, 2023. https://doi.org/10.48550/arXiv.2303.08774.
OpenAI. “Guide: Prompt Engineering.” OpenAI Documentation. Accessed November 24, 2023. https://platform.openai.com/docs/guides/prompt-engineering.
“Pandas.” Accessed November 28, 2023. https://pandas.pydata.org/.
Radford, Alec, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, and others. “Language Models Are Unsupervised Multitask Learners.” OpenAI Blog 1, no. 8 (2019): 9.
Ryu, Chesung Justin. “The Silence of Jonah: A Postcolonial Reading of Jonah 4:1-11.” PhD Dis-sertation, University of California Berkley, 2008.
“Seaborn: Statistical Data Visualization.” Accessed November 28, 2023. https://seaborn.pydata.org/.
Sheng, Emily, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. “The Woman Worked as a Babysitter: On Biases in Language Generation.” arXiv, October 23, 2019. https://doi.org/10.48550/arXiv.1909.01326.
Suchocki, Marjorie. God, Christ, Church: A Practical Guide to Process Theology. New, rev. Ed. A Herder & Herder Book. New York: Crossroad, 1989.
Touvron, Hugo, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Ni-kolay Bashlykov, et al. “Llama 2: Open Foundation and Fine-Tuned Chat Models.” arXiv, Ju-ly 19, 2023. https://doi.org/10.48550/arXiv.2307.09288.
Tunstall, Lewis, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Bel-kada, Shengyi Huang, et al. “Zephyr: Direct Distillation of LM Alignment.” arXiv, October 25, 2023. https://doi.org/10.48550/arXiv.2310.16944.
Turing, A. M. “Computing Machinery and Intelligence.” Mind LIX, no. 236 (October 1, 1950): 433–60. https://doi.org/10.1093/mind/LIX.236.433.
Vader Sentiment Analysis. “VaderSentiment Github Project.” Accessed November 28, 2023. https://github.com/cjhutto/vaderSentiment.
Young, Edward. Night Thoughts On Life, Death, and Immortality. Philadelphia, PA: James Rice & Company, 1800.
Zheng, Lianmin, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, et al. “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena.” arXiv, October 15, 2023. https://doi.org/10.48550/arXiv.2306.05685.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 A.G. Elrod
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Counting from volume 9 (2024), articles published in HIPHIL Novum are licensed under Attribution-ShareAlike 4.0 International (CC BY-SA 4.0). The editorial board may accept other Creative Commons licenses for individual articles, if required by funding bodies e.g. the European Research Council. With the publication of volume 9, authors retain copyright to their articles and give Hiphil Novum the right to the first publication. The authors retain copyright to earlier versions of the articles, such as the submitted and the accepted manuscript. Authors and readers may use, reuse, and build upon the published work, use it for text or data mining or for any other lawful purpose, as long as appropriate attribution is maintained.
Articles in volumes 1-8 are not licensed under Creative Commons. In these volumes, all rights are reserved to the authors of the articles respectively. This implies that readers can download, read, and link to the articles, but they cannot republish the articles. Authors may post the published version of their article to their personal website, institutional repository, or a repository required by their funding agency as a part of a green open access policy.
