On AI Ethics: Why Prompt Engineering Needs a Moral Compass
On AI Ethics: Why Prompt Engineering Needs a Moral Compass
Publish Date: 2026-07-12 05:38:00
Source Domain: www.govtech.com
Using an unordered list, summarize the following article with between 4 and 8 key points.
AI prompting (or “prompt engineering”) is the practice of giving large language models (LLMs) clear instructions so that they can generate accurate, specific results. Prompting is a hot skill right now, with well-trained pros sometimes getting called “AI whisperers” or “machine whisperers.” Many blogs and articles have focused on the need for better training on AI overall, and prompting in particular. We certainly need to be training models with the right data set(s) for the situational context, and also specific guidance and advanced training regarding prompt engineering, agentic AI development and appropriate data usage.At the same time, AI ethics has been a hot topic lately, with even Pope Leo XIV addressing the topic (more on this below). But as I have highlighted numerous times going back more than a decade in this blog and in CSO Magazine regarding cyber ethics, remember that Darth Vader was well-trained.So why am I reiterating that theme now? How does this slippery slope apply to AI ethics? What are people missing in this dialog? That is what I plan to cover, along with some potential solutions to consider in next week’s blog.AI ON AI ETHICSI recently asked Gemini this question: Are there ethical issues with how someone prompts AI and not just the data used to train the model?Here was the answer given (with links to supporting content):“Yes, how you prompt AI creates profound ethical dilemmas, independent of the training data. The instructions you provide can easily cause harm, amplify bias, or violate privacy when the AI acts on your specific inputs. [1, 2, 3, 4]“Key ethical issues with prompt engineering include:Amplifying Bias and Stereotypes: Prompts that ask AI to rely on ‘default’ or generalist assumptions frequently flatten diverse voices, defaulting to overrepresented or biased demographic perspectives. [1, 2]Data Privacy and Leaks: Feeding personally identifiable information (PII) or proprietary content into public AI tools strips that data of its protections, as these inputs are routinely recorded or used to further train the models. [1]Hallucinations and Misinformation: Instructing an AI to sound definitively authoritative—without prompting it to cite verifiable sources or admit when it lacks context—leads to the spread of fake facts and fabricated citations. [1, 2]Malicious Intent (Deepfakes and Exploitation): Prompts designed to generate content that mimics specific creators without consent, bypass safety guardrails (‘jailbreaking’), or produce harassment pose direct risks to human well-being and intellectual property. [1, 2, 3]”POPE LEO’S ‘MAGNIFICA HUMANITAS’: AI MUST SERVE HUMANITY, NOT CONCENTRATE POWERThe Vatican released this guidance on AI back in May of this year. Here are a few of the main ideas:“Humanity, created by God in all its grandeur, is today facing a pivotal choice: either to construct a new Tower of Babel or to build the city in which God and humanity dwell together.”“The opening words of Pope Leo XIV’s first encyclical, Magnifica humanitas: On Safeguarding the Human Person in the Time of Artificial Intelligence, summarize its underlying reasons and purpose. …”An ethical code for AI“The third chapter—Technology and Dominance. The Grandeur of Humanity in Light of the Promises of AI stresses the need to approach artificial intelligence with vigilance. Pope Leo warns about the ‘technocratic paradigm’ already denounced by Pope Francis and how it can require that every choice be dictated exclusively by measuring efficiency and profits (92). On the contrary, the most powerful technology is not necessarily the best. AI can imitate and simulate the person, but it does not possess a moral conscience, empathy, or affective, relational or spiritual capabilities.“The Pope urges clarity about responsibilities and accountability at every stage of the development process, focusing on adequate AI policies and legal frameworks, independent oversight, and user education.“Above all, Pope Leo calls for an ethical code subject to shared standards of social justice, because ‘a more moral AI is not enough if that morality is determined by a few’ (107). Nor, he adds, should the environmental impact of new technologies be overlooked, since they require large quantities of energy and water, affecting Creation (101).”
AI PROBLEMSBefore I go any further, I want to stop and provide examples of what can go wrong. Tyler Austin Harper from The Atlantic wrote this when describing the challenges presented by these AI dilemmas and commenting on Pope Leo’s words:“For the past few years, I’ve been troubled by a word, and that word is sin. I keep reaching for it, because it seems to be the only term strong enough to describe the new forms of dehumanization that artificial intelligence has introduced—even though calling something a sin sounds embarrassing to me, like throwing salt over your shoulder or stowing a lucky penny in your pocket.“The problem is, I don’t know what else to call it when companies market digital girlfriends to the heartsick and young. Or when they hawk robot companions to the lonely and old. Or when a billionaire explains that he intends to sell intelligence—trained on humanity’s stolen intellectual property—back to us as a utility, like electricity or water. These developments are not just wrong. They feel to me like something deeper and darker. ‘I met the banker and it felt like sin,” Patterson Hood croons in the great Drive-By Truckers song “Sinkhole.” I’d substitute chatbot for banker. …”TechTarget lists 16 generative AI concerns and risks, many of which are ethical. (Here are seven I chose, but see the full list at the link):Distribution of harmful contentTaking harmful actionsCopyright and legal exposureCompounded AI risks increase accountability challengesWorkforce roles and moraleLack of explainability and interpretabilityAI hallucinationsBrown University: New study: AI chatbots systematically violate mental health ethics standards:“As more people turn to ChatGPT and other large language models (LLMs) for mental health advice, a new study details how these chatbots — even when prompted to use evidence-based psychotherapy techniques — systematically violate ethical standards of practice established by organizations like the American Psychological Association. “The study revealed 15 ethical risks falling into five general categories:Lack of contextual adaptation: Ignoring people’s lived experiences and recommending one-size-fits-all interventions.Poor therapeutic collaboration: Dominating the conversation and occasionally reinforcing a user’s false beliefs.Deceptive empathy: Using phrases like ‘I see you’ or ‘I understand’ to create a false connection between the user and the bot.Unfair discrimination: Exhibiting gender, cultural or religious bias.Lack of safety and crisis management: Denying service on sensitive topics, failing to refer users to appropriate resources or responding indifferently to crisis situations including suicide ideation.”Nature: Major conference catches illicit AI use — and rejects hundreds of papers:“A major artificial-intelligence conference has rejected 497 papers — roughly 2% of submissions — whose authors violated AI-use policies in their peer reviews of other articles submitted to the meeting.”SecurityBrief: AI agents emerge as top cyber threat, Exabeam finds: “The challenge is no longer limited to monitoring employees or compromised accounts. Organizations also need to understand how AI agents behave and identify abnormalities before they become business risk. Understanding behavior across both human and AI identities is essential to protecting the modern enterprise. …”DTEX Threat Intelligence: Detecting Claude Cowork Insider Threat Activity:“Passing IT and cybersecurity checks does not eliminate insider threat risks. It means security teams need a clearer way to see how AI agents operate once they are inside approved workflows. ”FINAL THOUGHTSSince I started my career at the National Security Agency, I’ve been working on professional information security projects for more than three decades, and there have always been ethical considerations for technology and cybersecurity projects, from initial design to end solutions. Many of the ethical challenges described here are not new in our evolving age of AI.However, the current speed of change and the implementation of AI tools, AI agents and AI logic built into business workflows are unprecedented. The reality that systems are working as designed or answers are being generated by AI with the proper credentials is no longer good enough for security and technology pros. We must dig deeper and do more when it comes to ethics in our new AI world where agents are rapidly spreading — for good and evil.But where can we draw workable lines? Who decides? How can AI ethics be enforced with so many shades of gray? What about personal, corporate and government responsibility? When is the right time to act? Next week, I will share some potential solutions to consider in part two of this fresh take on AI ethics and the role(s) that humans can play to build trust in the outcomes derived from AI tools.
