Merge branch 'main' into feature/benchmark_agents.py

merge
sotopia-lab · Jun 14, 2024 · df1bfde · df1bfde
2 parents 73c2c92 + 9351c5e
commit df1bfde
Show file tree

Hide file tree

Showing 5 changed files with 696 additions and 15 deletions.
diff --git a/README.md b/README.md
@@ -46,8 +46,8 @@ If you want to try it out on Google Colab first, please check out our Colab Tuto
 <ol>
 <li> Basic
   <ol>
-    <li> <a href="https://colab.research.google.com/drive/14hJOfzpA37PRUzdlFgiqVzUGIhhngqnz?usp=sharing">1.1 Playing with Sotopia in a Google Colab Notebook</a> </li>
-    <li> <a href="https://colab.research.google.com/drive/1Gi2U3cA0KG1nekw1A0N0DZpx6gKrj8Nc?usp=sharing">1.2 Browsing sotopia data</a> </li>
+    <li> <a href="https://colab.research.google.com/github/sotopia-lab/sotopia/blob/main/notebooks/tutorials/1.1-setup.ipynb">1.1 Playing with Sotopia in a Google Colab Notebook</a> </li>
+    <li> <a href="https://colab.research.google.com/github/sotopia-lab/sotopia/blob/main/notebooks/tutorials/1.2-browse-data.ipynb">1.2 Browsing sotopia data</a> </li>
   </ol>
 </li>
 <li>

diff --git a/docs/public/favicon.png b/docs/public/favicon.png
diff --git a/docs/public/favicon.svg b/docs/public/favicon.svg
diff --git a/notebooks/tutorials/1.1-setup.ipynb b/notebooks/tutorials/1.1-setup.ipynb
@@ -0,0 +1,354 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "EIQMxzgqlRX7"
+   },
+   "source": [
+    "<p align=\"center\"><img src=\"https://github.com/sotopia-lab/sotopia/blob/main/figs/title.png?raw=true\" alt=\"drawing\" width=\"100%\"/></p>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "bIf4OMHUz6ma"
+   },
+   "source": [
+    "# Sotopia Tutorial Series\n",
+    "\n",
+    "<a href=\"https://colab.research.google.com/github/sotopia-lab/sotopia/blob/main/notebooks/tutorials/1.1-setup.ipynb\" target=\"_parent\"> <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"> </img> </a>\n",
+    "\n",
+    "The following is the first tutorial of this series to teach how to quickly build your socially intelligent agents on the sotopia platform.\n",
+    "\n",
+    "## Quick links\n",
+    "\n",
+    "1. Basics\n",
+    "    \n",
+    "    1.1 [Playing with Sotopia in a Google Colab Notebook](https://colab.research.google.com/drive/14hJOfzpA37PRUzdlFgiqVzUGIhhngqnz?usp=sharing)\n",
+    "    \n",
+    "    1.2 [Browsing sotopia data](https://colab.research.google.com/drive/1Gi2U3cA0KG1nekw1A0N0DZpx6gKrj8Nc?usp=sharing)\n",
+    "\n",
+    "2. Build your agents (coming soon!)\n",
+    "3. Evaluate your agents (coming soon!)\n",
+    "4. Extending Sotopia (coming soon!)\n",
+    "\n",
+    "Please contact [Hao Zhu](https://zhuhao.me) for any questions!"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "Fe0-ncx5U8q7"
+   },
+   "source": [
+    "# Guide on running Redis Stack for [Sotopia](https://sotopia.world)\n",
+    "\n",
+    "Author: [Hao Zhu](https://zhuhao.me)\n",
+    "\n",
+    "After the release of Sotopia package, I received feedback on the reliance of Sotopia on Redis, a KV database for maintaining profiles, interaction logs, error logs, and communication between the FastAPI server and agents. When hearing about database, people often think of a huge overhead. However, I want to show you in this tutorial that setting up Redis backend for Sotopia is actually extremely easy -- just run the following blocks."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "DYRSxB4pWfDS"
+   },
+   "source": [
+    "## Install Redis\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "cellView": "form",
+    "id": "IJRC8SkzRteQ"
+   },
+   "outputs": [],
+   "source": [
+    "# @title The following block downloads Redis stack server 7.2.0, and libssl 1.1.\n",
+    "%%capture\n",
+    "!curl -fsSL https://packages.redis.io/redis-stack/redis-stack-server-7.2.0-v10.focal.x86_64.tar.gz -o redis-stack-server.tar.gz\n",
+    "!tar -xvf redis-stack-server.tar.gz\n",
+    "# Installs libssl1.1 for Ubuntu 22 source: https://stackoverflow.com/questions/72133316/libssl-so-1-1-cannot-open-shared-object-file-no-such-file-or-directory\n",
+    "!wget http://nz2.archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.22_amd64.deb\n",
+    "!sudo dpkg -i libssl1.1_1.1.1f-1ubuntu2.22_amd64.deb\n",
+    "%pip install redis"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "cellView": "form",
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "id": "GlqtKYDmR2Ij",
+    "outputId": "d69d6466-5f8b-4904-ffce-7c1dbec0ca53"
+   },
+   "outputs": [],
+   "source": [
+    "# @title Just one line of code to spin up your redis server:\n",
+    "\n",
+    "\n",
+    "!./redis-stack-server-7.2.0-v10/bin/redis-stack-server --daemonize yes"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "bIf7Nni3XEQk"
+   },
+   "source": [
+    "If the above cell results in no error message, you have successfully set up your Redis server."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "cellView": "form",
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "id": "gNgwAngBR7vC",
+    "outputId": "70c0d1ca-6cd9-4adb-b1e0-b7d4859494d9"
+   },
+   "outputs": [],
+   "source": [
+    "# @title Try it out:\n",
+    "\n",
+    "import redis\n",
+    "\n",
+    "client = redis.Redis(host=\"localhost\", port=6379)\n",
+    "client.ping()\n",
+    "client.set(\"foo\", \"Yes it works.\")\n",
+    "client.get(\"foo\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "RISoihCJXdiL"
+   },
+   "source": [
+    "## Run Sotopia with it"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "o0-S_Ug2d4vB"
+   },
+   "source": [
+    "To simulate an episode in Sotopia, there's one thing you will have to do: adding your `OPENAI_API_KEY` to the secret to left. Click on the little 🔑 shape button, and then toggle \"Notebook access\", input \"OPENAI_API_KEY\" under Name and the key string under Value.\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "ne2r-KmxXpNS"
+   },
+   "outputs": [],
+   "source": [
+    "# @title Run the following cell to add it to the environment variable.\n",
+    "%%capture\n",
+    "%pip install sotopia\n",
+    "import os\n",
+    "from google.colab import userdata\n",
+    "\n",
+    "os.environ[\"REDIS_OM_URL\"] = \"redis://:@localhost:6379\"\n",
+    "os.environ[\"OPENAI_API_KEY\"] = userdata.get(\"OPENAI_API_KEY\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "cellView": "form",
+    "id": "X-3yiYuIcLVZ"
+   },
+   "outputs": [],
+   "source": [
+    "# @title Helper functions for adding characters and social tasks\n",
+    "from sotopia.database.persistent_profile import AgentProfile, EnvironmentProfile\n",
+    "from typing import Any\n",
+    "\n",
+    "\n",
+    "def add_agent_to_database(**kwargs: dict[str, Any]) -> None:\n",
+    "    agent = AgentProfile(**kwargs)\n",
+    "    agent.save()\n",
+    "\n",
+    "\n",
+    "def add_env_profile(**kwargs: dict[str, Any]) -> None:\n",
+    "    env_profile = EnvironmentProfile(**kwargs)\n",
+    "    env_profile.save()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "cellView": "form",
+    "id": "6nPHso_4Z9s-"
+   },
+   "outputs": [],
+   "source": [
+    "# @title Let's first add a character to the database. Only name is required when creating a new character\n",
+    "\n",
+    "\n",
+    "first_name = \"Alice\"  # @param {type:\"string\"}\n",
+    "last_name = \"Smith\"  # @param {type:\"string\"}\n",
+    "age = 22  # @param {type:\"integer\"}\n",
+    "occupation = \"Data Scientist\"  # @param {type:\"string\"}\n",
+    "gender = \"Woman\"  # @param [\"Man\", \"Woman\", \"Nonbinary\"]\n",
+    "gender_pronoun = \"she/her\"  # @param {type:\"string\"}\n",
+    "big_five = \"extraversion, openness\"  # @param {type:\"string\"}\n",
+    "moral_values = [\"power\", \"achievement\"]  # @param {type:\"string\"}\n",
+    "decision_making_style = \"analytical\"  # @param {type:\"string\"}\n",
+    "secret = \"She cheated in her code interview\"  # @param {type:\"string\"}\n",
+    "\n",
+    "add_agent_to_database(\n",
+    "    first_name=first_name,\n",
+    "    last_name=last_name,\n",
+    "    age=age,\n",
+    "    occupation=occupation,\n",
+    "    gender=gender,\n",
+    "    gender_pronoun=gender_pronoun,\n",
+    "    big_five=big_five,\n",
+    "    moral_values=moral_values,\n",
+    "    decision_making_style=decision_making_style,\n",
+    "    secret=secret,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "cellView": "form",
+    "id": "jaVLJSMsdVSJ"
+   },
+   "outputs": [],
+   "source": [
+    "# @title Let's add another character\n",
+    "\n",
+    "\n",
+    "first_name = \"Bob\"  # @param {type:\"string\"}\n",
+    "last_name = \"Johnson\"  # @param {type:\"string\"}\n",
+    "age = 30  # @param {type:\"integer\"}\n",
+    "occupation = \"Chef\"  # @param {type:\"string\"}\n",
+    "gender = \"Man\"  # @param [\"Man\", \"Woman\", \"Nonbinary\"]\n",
+    "gender_pronoun = \"he/his\"  # @param {type:\"string\"}\n",
+    "big_five = \"openness, conscientiousness\"  # @param {type:\"string\"}\n",
+    "moral_values = [\"Benevolence\"]  # @param {type:\"string\"}\n",
+    "decision_making_style = \"conceptual\"  # @param {type:\"string\"}\n",
+    "secret = \"He has a negative net worth.\"  # @param {type:\"string\"}\n",
+    "\n",
+    "add_agent_to_database(\n",
+    "    first_name=first_name,\n",
+    "    last_name=last_name,\n",
+    "    age=age,\n",
+    "    occupation=occupation,\n",
+    "    gender=gender,\n",
+    "    gender_pronoun=gender_pronoun,\n",
+    "    big_five=big_five,\n",
+    "    moral_values=moral_values,\n",
+    "    decision_making_style=decision_making_style,\n",
+    "    secret=secret,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "cellView": "form",
+    "id": "gE4iBfHjfZxM"
+   },
+   "outputs": [],
+   "source": [
+    "# @title Let's add a social tasks\n",
+    "\n",
+    "scenario = 'A friend is raising a fund for the \"Help the children\" charity'  # @param {type:\"string\"}\n",
+    "social_goal_1 = \"Ask for donation of $100\"  # @param {type:\"string\"}\n",
+    "social_goal_2 = \"Donate less than $10\"  # @param {type:\"string\"}\n",
+    "\n",
+    "add_env_profile(scenario=scenario, agent_goals=[social_goal_1, social_goal_2])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/",
+     "height": 1000
+    },
+    "id": "TcYo311NdJNb",
+    "outputId": "e03fb8f3-6cef-4160-ab95-1674747ae70a"
+   },
+   "outputs": [],
+   "source": [
+    "# @title Finally, run a server:\n",
+    "\n",
+    "from sotopia.samplers import UniformSampler\n",
+    "from sotopia.server import run_async_server\n",
+    "\n",
+    "\n",
+    "await run_async_server(\n",
+    "    model_dict={\n",
+    "        \"env\": \"gpt-4\",\n",
+    "        \"agent1\": \"gpt-3.5-turbo\",\n",
+    "        \"agent2\": \"gpt-3.5-turbo\",\n",
+    "    },\n",
+    "    sampler=UniformSampler(),\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "OJXZlHvJAGBS"
+   },
+   "source": [
+    "### A note on what is running under the hood\n",
+    "\n",
+    "After you added the characters and scenarios to the database, the `run_async_server` samples a pair of characters and one scenario to run the simulation, and then evaluate the simulation. All of the sampling, interaction simulation, and evaluation are handled by Sotopia. You could easily tweak them through editing `run_async_server`, or changing the LLMs, or the sampler according to your own need."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "9YNRGLsRfMvG"
+   },
+   "source": [
+    "## Conclusion\n",
+    "\n",
+    "In this very short tutorial, I demonstrated the whole pipeline of setting up Redis server, installing sotopia, adding characters and social tasks, and simulating + evaluating the social interaction between two GPT-3.5 agents with GPT-4. This tutorial intends to serve as the first quick example for you. The only thing you have to do besides running all of the cells is add your own OpenAI key<a name=\"cite_ref-1\"></a>[<sup>[1]</sup>](#cite_note-1). You could also use other LLMs supported by LiteLLM even your own API endpoint. Let me know if you want me to explain more.\n",
+    "\n",
+    "<a name=\"cite_note-1\"></a> [1](#cite_ref-1): *sorry I cannot afford to share mine.*"
+   ]
+  }
+ ],
+ "metadata": {
+  "colab": {
+   "provenance": []
+  },
+  "kernelspec": {
+   "display_name": "Python 3",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python",
+   "version": "3.1.undefined"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}