LangChain Zoomcamp Updates, Generative AI Ethics, and the Future of LLMs

Diving Deep: Prompting Techniques, AI in Content Creation, and the Ever-Evolving World of Large Language Models

What’s up, everyone!

Thank you to everyone who joined the LangChain ZoomCamp on Friday.

Don't worry if you missed it; I’ve got the recording for you. I’ll stick to sending the links directly to the Zoom recordings; here’s the link to the Zoom recording where you can watch or download the videos.

Here's a quick recap of what we covered:

• 🚀 NeurIPS Challenge: I’m part of a small team trying hard to participate in the NeurIPS LLM efficiency challenge, using DeciLM-6B as our base model. I talked about how I’m curating a dataset for the challenge.

• 🧠 Prompting Techniques: I introduced the distinctions between zero-shot and few-shot prompting. Additionally, I shared insights into the "chain of thought" method, a technique that guides the model through a series of reasoning steps. I also touched upon an augmented version of this method, which I've termed "take a deep breath."

• 📖 Example Selectors based on Retrieval: I showcased a system that employs embeddings to search a database for examples similar to a user's query. This system beautifully balances relevance and diversity in its results.

• 🤔 Diving into Reasoning Paths & Self-Consistency: We explored two intriguing prompting techniques. While the "chain of thought" method mimics human reasoning, the "self-consistency" approach samples diverse reasoning paths to determine the most consistent answer.

• 📚 Course Updates & Deep Learning Simplified: I'm grateful for all the feedback on my course. For those interested, I've introduced a new course on deep learning for image classification via the LinkedIn Learning platform.

As always, your feedback for the LangChain Zoomcamp is invaluable to me.

Where to find me (virtually) this week

I’ll be talking about Generative AI in a few live sessions this week.

• On October 24th, I’ll join Andreas Welsch on his LIVE show called “What’s the Buzz” to talk about Retrieval Augmented Generation. You can find the details for that here.

• On October 25th, I will be leading a technical webinar about Diffusion models. Specifically, I will discuss the Deci Diffusion model developed by my colleagues. If you're interested, you can register for the webinar using this link.

• On October 26th, I’ll join Andreas Kretz for his podcast. We’ll discuss Generative AI, Prompt Engineering, LangChain, LlamaIndex, and more!

Ethics and Responsibility in Generative AI

Thanks to the Generative AI World Summit for sponsoring this session. You can register for the conference with the discount code 'harpreet' for 75$ off your ticket price.

In a panel discussion I hosted with several experts and practitioners in Generative AI, we delved deep into the ethical challenges and potential misuse of this powerful technology. Generative AI, with its capability to craft lifelike content, brings forth pressing concerns related to misinformation and deepfakes.

Here are my key Insights from the Discussion:

• 🛡️ Combatting Misuse: We discussed strategies like watermarking and blockchain technology to trace content origins and detect tampering. Emphasis was also placed on robust data management to curb misinformation.

• 🏢 Corporate Responsibility: The onus is on organizations to rigorously test and validate their AI outputs, ensuring they align with ethical and legal standards.

• ⚖️ Legislative Measures: While regulating consumer tech poses challenges, the need for verifying AI outputs to ensure ethical and legal compliance was underscored.

• 🌍 Harnessing AI for Good: We highlighted the significance of representative training data and evaluation metrics to ensure fairness and reduce biases, allowing generative AI to benefit society.

• 🔧 Role of Data Scientists & Engineers: Tools like the Lancetest library were spotlighted as essential for testing. The responsibility of professionals in the field to prioritize ethics was emphasized.

• 🤝 Open-Source Community's Role: This community stands as a pillar for the responsible development of generative AI. Engaging with open-source projects offers a fantastic avenue for learning and contributing to the ethical advancement of AI.

As generative AI's potential expands, our commitment to ethical considerations and responsible practices must grow in tandem. Our discussion provided a comprehensive overview of the challenges and strategies in this domain, emphasizing our collective responsibility in navigating the intricate world of generative AI.

Get a shirt, support the newsletter

I keep all my content freely available by partnering with brands for sponsorships.

Lately, the pipeline for sponsorships has been a bit dry, so I launched a t-shirt line to gain community support.

You can check out the designs I made here.

✨ Blog of the Week

The 2023 Kaggle AI Report was released and its a compilation of essays from the world's largest data science and machine learning community.

The report offers a deep dive into the advancements and challenges of generative AI. This report is a testament to the collective knowledge of over 15 million Kaggle members, who rigorously evaluate and share insights on the latest in AI and ML.

Key Takeaways:

• 🌍 Global AI Progress: The AI landscape has witnessed remarkable progress, with models like ChatGPT, Llama, and PaLM leading the charge. The global spread of AI knowledge has seen experts from diverse backgrounds contributing to the field.

• 🧠 Generative AI Insights: The report delves into generative AI, emphasizing its rapid evolution over the past two years. This domain, powered by generative adversarial networks (GANs) and large language models (LLMs), is revolutionizing content creation across text, images, and music.

• 🚀 Notable Essays: The award-winning essay by Trushant Kalyanpur charts the evolution of generative AI from 2021 to 2023, highlighting innovations like GPT4, DALL-E, and ChatGPT. Another essay by Yuqi Liu traces generative AI's historical and societal significance, exploring models like DALL·E 2.

🛠️ GitHub Gems

There were two libraries that were discussed during the panel discussion:

• LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). 📚 Extracts signals from prompts & responses, ensuring safety & security. 🛡️ Features include text quality, relevance metrics, & sentiment analysis. 📊 A comprehensive tool for LLM observability.

• LangTest:Generate & run over 50 test types on the most popular NLP frameworks & tasks with 1 line of code. Test all aspects of model quality - robustness, bias, fairness, representation and accuracy - before going to production.

📰 Industry Pulse

🌍 Could AI be the key to breaking down language barriers in content creation? Voice AI platform, ElevenLabs, has unveiled its groundbreaking AI Dubbing feature, marking a significant stride in the company's mission to eliminate language barriers and ensure universal content accessibility. This innovative tool can swiftly translate spoken content into various languages while retaining the original speaker's voice and nuances.

🤖 Is AI the future of multilingual communication for politicians? New York City Mayor Eric Adams has stirred controversy by using artificial intelligence to generate robocalls in multiple languages, including Mandarin and Yiddish, without disclosing that he only speaks English. These AI-generated calls were intended to promote city hiring events and reach the diverse population of New York.

🤔 Is AI the new weapon in the world of propaganda and disinformation?Governments and political entities globally are harnessing the power of artificial intelligence to manipulate public opinion. A report by Freedom House reveals that AI is being used to generate texts, images, and videos in various languages, without disclosing the AI-generated content. This has raised significant ethical concerns, especially as the line between genuine and AI-generated content becomes increasingly blurred.

🌐 Could AI be the bridge to connect India's linguistic diversity? Prime Minister Narendra Modi has officially unveiled 'Bhashini', an innovative AI-driven program initiated by the Indian government in 2022. This program aims to bridge the linguistic divide within India, a nation known for its vast linguistic diversity. The 2011 Census highlighted that India boasts 121 major languages spoken by at least 10,000 individuals each. However, many of these languages lack adequate online representation, hindering the growth of a comprehensive digital economy.

🌐 Is the future of language understanding AI-driven? In collaboration with Peng Cheng Laboratory, Baidu has unveiled 'PCL-BAIDU Wenxin' or 'ERNIE 3.0 Titan', a state-of-the-art AI-based language model with a whopping 260 billion parameters. This model is designed to bridge the linguistic gap by training on vast unstructured data and a massive knowledge graph. ERNIE 3.0 Titan is the world's first knowledge-enhanced multi-hundred billion parameter model and the largest Chinese singleton model.

🔍 Research Refined

This weeks paper is "How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advances" authored by Zihan Zhang, Meng Fang, Ling Chen, Mohammad-Reza Namazi-Rad, and Jun Wang from various universities.

🌐 How do modern AI models keep up with our rapidly evolving world?

LLMs are increasingly relied upon to process and understand vast amounts of textual data. Trained on vast corpora, including sources like Wikipedia, Books, and Github, they can store immense amounts of world knowledge in their parameters. This enables LLMs to be foundational models for various NLP tasks. They can directly perform in-context learning or undergo fine-tuning for domain-specific applications. However, the dynamic nature of our world contrasts with the static nature of deployed LLMs.

• 📚 LLMs, once trained, are static. They lack mechanisms to self-update or adapt to a continuously changing environment.

• 🌍 The static nature of LLMs means that the knowledge they hold can quickly become obsolete. This poses challenges, especially when models produce unreliable outputs for tasks that require up-to-date knowledge.

• 🔄 There's an increasing demand to ensure LLMs remain aligned with the ever-changing world knowledge post-deployment. This is crucial as numerous users and applications depend on them. However, re-training these models with new information often proves infeasible.

The paper presents a taxonomy of methods to align Large Language Models (LLMs) with the ever-changing world knowledge. These methods are categorized into two main approaches: Implicit and Explicit.

Let's break down each of these.

Implicitly Align LLMs with World Knowledge:

• This approach is based on the capability of LLMs to implicitly memorize knowledge due to their large number of parameters. After being pre-trained on massive corpora, LLMs can inherently capture and retain certain knowledge.

• Knowledge Editing (KE):

  • Given the costs and challenges of tuning LLMs, there's a push towards efficiently updating specific, localized, or fine-grained knowledge.

  • "Knowledge Editing" is emerging as a promising area of research. It aims to directly alter parameters corresponding to specific knowledge stored in pre-trained LLMs. Several referenced works propose methods to achieve this, such as targeting specific neurons or layers within the model.

• Continual Learning (CL):

  • CL is about training models on continuous streams of data over time, aiming to reduce the "catastrophic forgetting" of previously acquired knowledge.

  • The idea is to allow deployed LLMs to adapt to the world's changes without undergoing costly full re-training.

  • Methods under this category focus on aligning LLMs with current world knowledge using techniques like continual pre-training and continual knowledge editing.

Explicitly Align LLMs with World Knowledge:

Explicit methods primarily revolve around augmenting LLMs with external information sources through retrieval-augmented techniques or by incorporating external memory systems.

• While implicitly altering the knowledge stored in LLMs has been effective, it's uncertain whether this will affect the models' general abilities due to the inherent complexity of neural networks.

• An alternative approach is explicitly augmenting LLMs with the latest information sourced from various external sources. This can effectively adapt the models to new world knowledge without affecting the foundational knowledge of the original LLMs.

• Traditional retrieval-augmented methods, as cited in works by Karpukhin et al. (2020), Guu et al. (2020), Lewis et al. (2020), and others, typically involve joint training of a retriever and a language model in an end-to-end fashion. However, this approach can be challenging when applied to a deployed LLM, such as GPT-3.

• A recent direction in research involves equipping a fixed LLM with external memory, termed "memory-enhanced" methods.

That’s it for this one.

See you next week, and if there’s anything you want me to cover or have any feedback, shoot me an email.

Cheers,

Harpreet