From fcaad9975039c692e629e989fa04914bfa0f73aa Mon Sep 17 00:00:00 2001 From: Xuhui Zhou Date: Fri, 7 Jun 2024 08:51:06 -0700 Subject: [PATCH] Revert "Tutorial notebooks that can be directly opened in colab (#90)" This reverts commit 9351c5e3646187e8d142a54c4c55cd455ffed76e. --- README.md | 4 +- docs/public/favicon.png | Bin 4563 -> 0 bytes docs/public/favicon.svg | 19 +- notebooks/tutorials/1.1-setup.ipynb | 354 ---------------------- notebooks/tutorials/1.2-browse-data.ipynb | 334 -------------------- 5 files changed, 15 insertions(+), 696 deletions(-) delete mode 100644 docs/public/favicon.png delete mode 100644 notebooks/tutorials/1.1-setup.ipynb delete mode 100644 notebooks/tutorials/1.2-browse-data.ipynb diff --git a/README.md b/README.md index fb27cfc0a..528a8b85c 100644 --- a/README.md +++ b/README.md @@ -46,8 +46,8 @@ If you want to try it out on Google Colab first, please check out our Colab Tuto
  1. Basic
      -
    1. 1.1 Playing with Sotopia in a Google Colab Notebook
      2. -
    2. 1.2 Browsing sotopia data
      3. +
    3. 1.1 Playing with Sotopia in a Google Colab Notebook
      4. +
    4. 1.2 Browsing sotopia data
  2. diff --git a/docs/public/favicon.png b/docs/public/favicon.png deleted file mode 100644 index ef6b7c1af91506e6f374f9445914f5a48e27e39f..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 4563 zcmX9?c~p|y*9LP+aY#!Gr?8ve($G>6zAy*m{S^{CJLE|CWhhk(eL}?oU@+2_BrcaXP>>FXTO;jFCdi^zzP5WKs3&&-!BJ0s4xM*#qeYQG2Y6YK5<0I)0ltONW?ToIq;9QnmjLw2do z)9b?1V%CC#ATVf7I>TA@mvn`*>fG;1ES+&d^_w)%*(cJHltUUnT~#o9Wz|>=Pw{t= zItviQh(xbldTCZa3$K-WZ6n&_1-oEe6wpUu$^vV@wY-~Z^z!k)i8GYRTbgj3HP--O z@MGxts@)!yZdnizi3WPM8sH&sM( zkpFoQtH`=_pBN%Tq+;@6G1Iosl!#Gb3O)vU+9|p5YV*}eyG_k8=6j;K%#gHN8#x@u zzsS7&`Kqt}@Or*HQ922#7r1@npkJluseiAxHm9{Mt!ux62T`xy{-AjObV!!2Q zaU^B2$O8);i+Jsc4XCNLLD&y<52ivB*#{dFY6EIKu@*Jylm4+Wj*3K3>N!BmgVmr< zUZ4|;u026N9{D$ZaokB`oCX1jSf>$g23ZtJ}lm-p6ktEpi!C*kSW4U|;#DSyL}u2BzUEbZx3?B)Kj_li9A z9e0boQfN8I>FXq{t?5*ybs|rKW()FG`sPolv~@uGd`ZTOVXZl({1;`U|8ln$C}6NXpOReT*>QAj>hd;Tm)LNDd`bHp zUyArO)w$jJ77v=d>*TUV-b*yl(~3JM-yWTj1n5Q!W#Z5_29~3O@Vs4qJ8LUcUae?* z>RN2x_#9akB%p|%qIcnzmS>lv6^NkmhAHnePOj{Ad(wT3m8PEeSXtuEjFZO|`BFb0 zrJ9UQ{VlF4@qCb5qkNb9u>WBvgk;=^kwe%LgK&$qrXy>b>ROjgE^)pc$2cpqX8qL= zx_Nb7_O`%quuE)xc@z+O&H|hHG>U&ruZ~&=iE$nn{1utL_8%xcDZ^BcJC*}Xgqyh|+> z#}IBHYnvc%q<5+Z3=Jv{==09bLp0Av ze7Cp`B8D2+L$ktBzS-B*h1IwAyw84`1*ha1={FP(`#?A0@y z?T>qzV?xa_oR+b=-M~>mORjOry$LkmdmeC!a?{M$n<9$h>tC0|zsk;}bx*9KZM@(- zUmyIoj}}{Yki#1@xLkKDt&BK-IXJWi_W2N`sQkM-1T}EmfRiGUG3WtKjZUmJ2ix8I zs3K5w(!?#!yaJ-GCz%vnOMy76!NhT4K2!6Rh!=(1HJ)-xGvf`_pT=(iTH*yJH5sUD z6BFenZ^~6d(ixo%6X_?@qgI4>@U{=;GF(^JM_~o`AkJV|!gej-zlSPAdJ(kH)-2@X zEzEcUn)3Gn^;=w_&L_A%iJmXKkHaU8a=lg zf46`)H@&_D-A;Q^~;+3p7JT(=)` za@uw{1bzGY%1t>;dc>h=4N+@ggTegxUic(LI$mi?1H$lt7VLV+WfJN*D@W{e?ya9P z`%kuag!bfFp1h9e%{FkYZljT+xv2-(cH}Vt09sk*!uO~VO4p`k`Z?P=AoEAzy(6d? zIz9hO$be3J78h7|<6QR$ZW{B*g{6Opv50QSu>_)MlXFS{mUJ8s^!Nn>cQaN~tg3Kt8w zE4P#v6~-^}=zH2FEI&sAfZfi)!xI&!+yIlsQ64)t9UA_)Y&Fp9bPgec79`HNw7lCq z`{NbI?!$M&GDk!BQ7et=9isXCp2zL@UTfY%okh}yYr~UeEAFWLW$nIGucCxEP=K`u z_4nQS>W|Bjt;`O9#0Wj$#D&0=bE zvP}KT(%zJpa-~GCh3d1{FMJq=Vl%o4q8Xhw+YS9T4_E6XIv|x@M+c13n$Gj>iI0b{ zmJ>0I`n|#}2?0s+T4yk#q+Qf;=V;h9i?(}OEAX$+Q4^mBajObc!P{#=n8jv>)mlaxUGJlMUW;Jest&SYsyo|@_B!I;~sL{by-^!Uh72flXZ2)2)RoQWa< z=JjC2<~nxb<>{|6jBmb-&_?*)zQQn36@CVw6b ztp@Ja(D=qFq=8ZS=cQo8BXz1LNMfx(fAGp>^s@{41}oB-957uxK`o5!{(umP1t-^6kvaM z^!T{a4aST}LlzoyJH7bzvg{pR30|bZp6?rpE8HuLUH~{9K_NTqzqFSno`CxJ$g40z zDSuJbqXX;9E4e}Zt7{p?YX8W`eto2m77Fv@%gRMtqlw8o*)HXHxHBCVVv>=Q_}va` z^y8LQwDS-x(QXj>^{D#`qt!T za6nA3PwwRM;xlipQjDyDjR5YF+?#UlO9s?OK08B*dO4X2r=KtLq~`pw zB!d+JYM=BUO-G2xRW0vAooG!v+d|3%dy=lv2|-kpoF_l%3npJujD>Re}(0 z{Occ3pqz7>eEwHl<#i`#NyRbcA#OLXpD6jo7ms=^X!p@hsOD%!OVv;`nGL$cF`L(?J04u4V5Og2j zaM}@2cPF{4EQe0P9PnYR2y;bSt|a;S#la6Ob-=dN5`EMzr1|W|4r;&N7RHn}?u%@w zwjhVciViKRSQnoh+lKsdFfT$ztr{@YLFB)>3F=Cxlqbx;gsK0!=z$4hWF{~sTtaLM zPn>BdsOwb67=Oe*!u;S$LNZ-^%P z{9CeGbD_H&^|UKQl8vDfnYapL29W-h&VH>tmyQOSR|QBzN)MniT60wHOB}+C3)&&)WN;v_BFG2We5Kqm{_0w{j>VwmPvhRuca zpJZ)NJZKRr6Tp~R=}l}IHKtm-r9iUbBO6$FcE9G1tZ>Dp4iC^RfekDnt(GiO9FXGt zaI`(Ii+j7N4(;lk?n|^AGR9^XIC8l^cW3LcY!xBli?KI)X6oa~_;$jV3f;(7%ZiR! z9BIQkx}o2pC%|h9ht?S$XKQ6h2;!~UkHDS_%bydy_os%@!!_)o=OVmxbHQ?6t({qh zw|~8$IJ0vgFz;H`ywry6NPHOLVZ^^zS35iq84Diuh25OqFUi)?V-4`QQ=JYyol>Cq z+a=`wQXS7RCG4{|-=7%*e$z5OO7vs`vcOBYyTbOquF!OU4l^Ge^pj~a<;Pq@1E3F; zMu1GpMvh{{ou*~B8+}t>nbjQ-R>htxdQFWj?|iZHy}>G45Bab@Ch$&`@>Ff18RyP7 zrn$t%R+u*Nf9=_0HYefFOU)lkEt8dTi|y4wW3p&Yv!U^mZgIJZYyeU(<6G03?!w%1 z)h>;wZ1IF2QZS|rnR<$bbVtxwDRTaP%Ue(COrTw><&KT?P&MD4AfKKa%io6UOq5yl z*mP4(Nsk?=zbg88$uP(h)K|9t%~f7tV-~yg(kMP|Hk^Ng=N>faOf6eqc9mcG!_2v` z_#TVd>;;8PGbp4zZK{+{&=*Kab3kF4ca*KKxPYtxRO{}RL{hUbKHCl^_O+C!uena7 zgW!1bW$a!o5T j2dCw-W`}TC-_+AV1jht1=MPH$(g4n$zTi-6@00R>sS_AH diff --git a/docs/public/favicon.svg b/docs/public/favicon.svg index 1a4f0aff1..ed06047cb 100644 --- a/docs/public/favicon.svg +++ b/docs/public/favicon.svg @@ -1,7 +1,14 @@ - - - - - - + + + + + diff --git a/notebooks/tutorials/1.1-setup.ipynb b/notebooks/tutorials/1.1-setup.ipynb deleted file mode 100644 index f00e7d33b..000000000 --- a/notebooks/tutorials/1.1-setup.ipynb +++ /dev/null @@ -1,354 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "id": "EIQMxzgqlRX7" - }, - "source": [ - "

    \"drawing\"

    " - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "bIf4OMHUz6ma" - }, - "source": [ - "# Sotopia Tutorial Series\n", - "\n", - " \"Open \n", - "\n", - "The following is the first tutorial of this series to teach how to quickly build your socially intelligent agents on the sotopia platform.\n", - "\n", - "## Quick links\n", - "\n", - "1. Basics\n", - " \n", - " 1.1 [Playing with Sotopia in a Google Colab Notebook](https://colab.research.google.com/drive/14hJOfzpA37PRUzdlFgiqVzUGIhhngqnz?usp=sharing)\n", - " \n", - " 1.2 [Browsing sotopia data](https://colab.research.google.com/drive/1Gi2U3cA0KG1nekw1A0N0DZpx6gKrj8Nc?usp=sharing)\n", - "\n", - "2. Build your agents (coming soon!)\n", - "3. Evaluate your agents (coming soon!)\n", - "4. Extending Sotopia (coming soon!)\n", - "\n", - "Please contact [Hao Zhu](https://zhuhao.me) for any questions!" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "Fe0-ncx5U8q7" - }, - "source": [ - "# Guide on running Redis Stack for [Sotopia](https://sotopia.world)\n", - "\n", - "Author: [Hao Zhu](https://zhuhao.me)\n", - "\n", - "After the release of Sotopia package, I received feedback on the reliance of Sotopia on Redis, a KV database for maintaining profiles, interaction logs, error logs, and communication between the FastAPI server and agents. When hearing about database, people often think of a huge overhead. However, I want to show you in this tutorial that setting up Redis backend for Sotopia is actually extremely easy -- just run the following blocks." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "DYRSxB4pWfDS" - }, - "source": [ - "## Install Redis\n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "id": "IJRC8SkzRteQ" - }, - "outputs": [], - "source": [ - "# @title The following block downloads Redis stack server 7.2.0, and libssl 1.1.\n", - "%%capture\n", - "!curl -fsSL https://packages.redis.io/redis-stack/redis-stack-server-7.2.0-v10.focal.x86_64.tar.gz -o redis-stack-server.tar.gz\n", - "!tar -xvf redis-stack-server.tar.gz\n", - "# Installs libssl1.1 for Ubuntu 22 source: https://stackoverflow.com/questions/72133316/libssl-so-1-1-cannot-open-shared-object-file-no-such-file-or-directory\n", - "!wget http://nz2.archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.22_amd64.deb\n", - "!sudo dpkg -i libssl1.1_1.1.1f-1ubuntu2.22_amd64.deb\n", - "%pip install redis" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "GlqtKYDmR2Ij", - "outputId": "d69d6466-5f8b-4904-ffce-7c1dbec0ca53" - }, - "outputs": [], - "source": [ - "# @title Just one line of code to spin up your redis server:\n", - "\n", - "\n", - "!./redis-stack-server-7.2.0-v10/bin/redis-stack-server --daemonize yes" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "bIf7Nni3XEQk" - }, - "source": [ - "If the above cell results in no error message, you have successfully set up your Redis server." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "gNgwAngBR7vC", - "outputId": "70c0d1ca-6cd9-4adb-b1e0-b7d4859494d9" - }, - "outputs": [], - "source": [ - "# @title Try it out:\n", - "\n", - "import redis\n", - "\n", - "client = redis.Redis(host=\"localhost\", port=6379)\n", - "client.ping()\n", - "client.set(\"foo\", \"Yes it works.\")\n", - "client.get(\"foo\")" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "RISoihCJXdiL" - }, - "source": [ - "## Run Sotopia with it" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "o0-S_Ug2d4vB" - }, - "source": [ - "To simulate an episode in Sotopia, there's one thing you will have to do: adding your `OPENAI_API_KEY` to the secret to left. Click on the little 🔑 shape button, and then toggle \"Notebook access\", input \"OPENAI_API_KEY\" under Name and the key string under Value.\n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "ne2r-KmxXpNS" - }, - "outputs": [], - "source": [ - "# @title Run the following cell to add it to the environment variable.\n", - "%%capture\n", - "%pip install sotopia\n", - "import os\n", - "from google.colab import userdata\n", - "\n", - "os.environ[\"REDIS_OM_URL\"] = \"redis://:@localhost:6379\"\n", - "os.environ[\"OPENAI_API_KEY\"] = userdata.get(\"OPENAI_API_KEY\")" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "id": "X-3yiYuIcLVZ" - }, - "outputs": [], - "source": [ - "# @title Helper functions for adding characters and social tasks\n", - "from sotopia.database.persistent_profile import AgentProfile, EnvironmentProfile\n", - "from typing import Any\n", - "\n", - "\n", - "def add_agent_to_database(**kwargs: dict[str, Any]) -> None:\n", - " agent = AgentProfile(**kwargs)\n", - " agent.save()\n", - "\n", - "\n", - "def add_env_profile(**kwargs: dict[str, Any]) -> None:\n", - " env_profile = EnvironmentProfile(**kwargs)\n", - " env_profile.save()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "id": "6nPHso_4Z9s-" - }, - "outputs": [], - "source": [ - "# @title Let's first add a character to the database. Only name is required when creating a new character\n", - "\n", - "\n", - "first_name = \"Alice\" # @param {type:\"string\"}\n", - "last_name = \"Smith\" # @param {type:\"string\"}\n", - "age = 22 # @param {type:\"integer\"}\n", - "occupation = \"Data Scientist\" # @param {type:\"string\"}\n", - "gender = \"Woman\" # @param [\"Man\", \"Woman\", \"Nonbinary\"]\n", - "gender_pronoun = \"she/her\" # @param {type:\"string\"}\n", - "big_five = \"extraversion, openness\" # @param {type:\"string\"}\n", - "moral_values = [\"power\", \"achievement\"] # @param {type:\"string\"}\n", - "decision_making_style = \"analytical\" # @param {type:\"string\"}\n", - "secret = \"She cheated in her code interview\" # @param {type:\"string\"}\n", - "\n", - "add_agent_to_database(\n", - " first_name=first_name,\n", - " last_name=last_name,\n", - " age=age,\n", - " occupation=occupation,\n", - " gender=gender,\n", - " gender_pronoun=gender_pronoun,\n", - " big_five=big_five,\n", - " moral_values=moral_values,\n", - " decision_making_style=decision_making_style,\n", - " secret=secret,\n", - ")" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "id": "jaVLJSMsdVSJ" - }, - "outputs": [], - "source": [ - "# @title Let's add another character\n", - "\n", - "\n", - "first_name = \"Bob\" # @param {type:\"string\"}\n", - "last_name = \"Johnson\" # @param {type:\"string\"}\n", - "age = 30 # @param {type:\"integer\"}\n", - "occupation = \"Chef\" # @param {type:\"string\"}\n", - "gender = \"Man\" # @param [\"Man\", \"Woman\", \"Nonbinary\"]\n", - "gender_pronoun = \"he/his\" # @param {type:\"string\"}\n", - "big_five = \"openness, conscientiousness\" # @param {type:\"string\"}\n", - "moral_values = [\"Benevolence\"] # @param {type:\"string\"}\n", - "decision_making_style = \"conceptual\" # @param {type:\"string\"}\n", - "secret = \"He has a negative net worth.\" # @param {type:\"string\"}\n", - "\n", - "add_agent_to_database(\n", - " first_name=first_name,\n", - " last_name=last_name,\n", - " age=age,\n", - " occupation=occupation,\n", - " gender=gender,\n", - " gender_pronoun=gender_pronoun,\n", - " big_five=big_five,\n", - " moral_values=moral_values,\n", - " decision_making_style=decision_making_style,\n", - " secret=secret,\n", - ")" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "id": "gE4iBfHjfZxM" - }, - "outputs": [], - "source": [ - "# @title Let's add a social tasks\n", - "\n", - "scenario = 'A friend is raising a fund for the \"Help the children\" charity' # @param {type:\"string\"}\n", - "social_goal_1 = \"Ask for donation of $100\" # @param {type:\"string\"}\n", - "social_goal_2 = \"Donate less than $10\" # @param {type:\"string\"}\n", - "\n", - "add_env_profile(scenario=scenario, agent_goals=[social_goal_1, social_goal_2])" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 1000 - }, - "id": "TcYo311NdJNb", - "outputId": "e03fb8f3-6cef-4160-ab95-1674747ae70a" - }, - "outputs": [], - "source": [ - "# @title Finally, run a server:\n", - "\n", - "from sotopia.samplers import UniformSampler\n", - "from sotopia.server import run_async_server\n", - "\n", - "\n", - "await run_async_server(\n", - " model_dict={\n", - " \"env\": \"gpt-4\",\n", - " \"agent1\": \"gpt-3.5-turbo\",\n", - " \"agent2\": \"gpt-3.5-turbo\",\n", - " },\n", - " sampler=UniformSampler(),\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "OJXZlHvJAGBS" - }, - "source": [ - "### A note on what is running under the hood\n", - "\n", - "After you added the characters and scenarios to the database, the `run_async_server` samples a pair of characters and one scenario to run the simulation, and then evaluate the simulation. All of the sampling, interaction simulation, and evaluation are handled by Sotopia. You could easily tweak them through editing `run_async_server`, or changing the LLMs, or the sampler according to your own need." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "9YNRGLsRfMvG" - }, - "source": [ - "## Conclusion\n", - "\n", - "In this very short tutorial, I demonstrated the whole pipeline of setting up Redis server, installing sotopia, adding characters and social tasks, and simulating + evaluating the social interaction between two GPT-3.5 agents with GPT-4. This tutorial intends to serve as the first quick example for you. The only thing you have to do besides running all of the cells is add your own OpenAI key[[1]](#cite_note-1). You could also use other LLMs supported by LiteLLM even your own API endpoint. Let me know if you want me to explain more.\n", - "\n", - " [1](#cite_ref-1): *sorry I cannot afford to share mine.*" - ] - } - ], - "metadata": { - "colab": { - "provenance": [] - }, - "kernelspec": { - "display_name": "Python 3", - "name": "python3" - }, - "language_info": { - "name": "python", - "version": "3.1.undefined" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} diff --git a/notebooks/tutorials/1.2-browse-data.ipynb b/notebooks/tutorials/1.2-browse-data.ipynb deleted file mode 100644 index ff500878c..000000000 --- a/notebooks/tutorials/1.2-browse-data.ipynb +++ /dev/null @@ -1,334 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "id": "EIQMxzgqlRX7" - }, - "source": [ - "

    \"drawing\"

    " - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "ocrtYaex1s0h" - }, - "source": [ - "# Sotopia Tutorial Series\n", - "\n", - " \"Open \n", - "\n", - "The following is the second tutorial of this series to teach how to quickly build your socially intelligent agents on the sotopia platform.\n", - "\n", - "## Quick links\n", - "\n", - "1. Basics\n", - " \n", - " 1.1 [Playing with Sotopia in a Google Colab Notebook](https://colab.research.google.com/drive/14hJOfzpA37PRUzdlFgiqVzUGIhhngqnz?usp=sharing)\n", - " \n", - " 1.2 [Browsing sotopia data](https://colab.research.google.com/drive/1Gi2U3cA0KG1nekw1A0N0DZpx6gKrj8Nc?usp=sharing)\n", - "\n", - "2. Build your agents (coming soon!)\n", - "3. Evaluate your agents (coming soon!)\n", - "4. Extending Sotopia (coming soon!)\n", - "\n", - "Please contact [Hao Zhu](https://zhuhao.me) for any questions!" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "Fe0-ncx5U8q7" - }, - "source": [ - "\n", - "# 1.2 Browsing [Sotopia](https://sotopia.world) data\n", - "\n", - "Author: [Hao Zhu](https://zhuhao.me)\n", - "\n", - "Running the code in this tutorial does not require familiarity of the redis setup. But if you have finished [Tutorial 1.1](https://colab.research.google.com/drive/14hJOfzpA37PRUzdlFgiqVzUGIhhngqnz?usp=sharing), you would have a better idea of the characters (in the format of `sotopia.database.AgentProfile`) and social tasks (in the format of `sotopia.database.EnvironmentProfile`). What you would be able to learn through this tutorial are:\n", - "\n", - "1. How to load all of the existing data from sotopia(-pi) experiments, including 40 characters, hundreds of scenarios, and tens of thousands of interaction episodes.\n", - "2. How to browse them through indexing based on two awesome libraries `redis-om` and `pydantic`." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "DYRSxB4pWfDS" - }, - "source": [ - "## Install Redis\n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "id": "IJRC8SkzRteQ" - }, - "outputs": [], - "source": [ - "# @title Download Redis: This code block is exactly the same as [Tutorial 1.1](https://colab.research.google.com/drive/14hJOfzpA37PRUzdlFgiqVzUGIhhngqnz?usp=sharing).\n", - "%%capture\n", - "!curl -fsSL https://packages.redis.io/redis-stack/redis-stack-server-7.2.0-v10.focal.x86_64.tar.gz -o redis-stack-server.tar.gz\n", - "!tar -xvf redis-stack-server.tar.gz\n", - "# Installs libssl1.1 for Ubuntu 22 source: https://stackoverflow.com/questions/72133316/libssl-so-1-1-cannot-open-shared-object-file-no-such-file-or-directory\n", - "!wget http://nz2.archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.22_amd64.deb\n", - "!sudo dpkg -i libssl1.1_1.1.1f-1ubuntu2.22_amd64.deb\n", - "%pip install redis" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "id": "GlqtKYDmR2Ij" - }, - "outputs": [], - "source": [ - "%%capture\n", - "# @title Download existing data in a single file, and launch redis\n", - "\n", - "!mkdir -p /content/redis-stack-server-7.2.0-v10/var/db/redis-stack\n", - "!curl -L https://huggingface.co/datasets/cmu-lti/sotopia-pi/resolve/main/dump.rdb?download=true --output /content/redis-stack-server-7.2.0-v10/var/db/redis-stack/dump.rdb\n", - "!./redis-stack-server-7.2.0-v10/bin/redis-stack-server --daemonize yes" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "bIf7Nni3XEQk" - }, - "source": [ - "If the above cell results in no error message, you have successfully set up your Redis server." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "gNgwAngBR7vC", - "outputId": "321eb7f8-29c1-4d84-aa9a-2f114a4f8f49" - }, - "outputs": [], - "source": [ - "# @title Your DB is now pre-populated with existing data:\n", - "\n", - "import redis\n", - "from redis import BusyLoadingError\n", - "import time\n", - "\n", - "client = redis.Redis(host=\"localhost\", port=6379)\n", - "while True:\n", - " try:\n", - " client.ping()\n", - " break\n", - " except BusyLoadingError:\n", - " time.sleep(1)\n", - "print(f\"Voila, you have loaded {client.dbsize()} data!\")" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "RISoihCJXdiL" - }, - "source": [ - "## Browse the data you have just loaded" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "id": "ne2r-KmxXpNS" - }, - "outputs": [], - "source": [ - "# @title First, install sotopia\n", - "%%capture\n", - "%pip install sotopia\n", - "import os\n", - "\n", - "os.environ[\"REDIS_OM_URL\"] = \"redis://:@localhost:6379\"" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "colab": { - "base_uri": "https://localhost:8080/", - "height": 353 - }, - "id": "ImjdFUlGGA4_", - "outputId": "df1abaac-fce7-4e14-fd84-0dc3d96d8521" - }, - "outputs": [], - "source": [ - "# @title Show me a random character\n", - "import random\n", - "from rich import print\n", - "\n", - "from sotopia.database import AgentProfile\n", - "\n", - "\n", - "all_character_pks = list(AgentProfile.all_pks())\n", - "pk = random.choice(all_character_pks)\n", - "\n", - "print(AgentProfile.get(pk))" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "colab": { - "base_uri": "https://localhost:8080/", - "height": 321 - }, - "id": "Su5EwfIPHQWp", - "outputId": "47e36153-fa91-4a34-cc5b-80dea2e516dd" - }, - "outputs": [], - "source": [ - "# @title Show me a random social task\n", - "from sotopia.database import EnvironmentProfile\n", - "\n", - "all_task_pks = list(EnvironmentProfile.all_pks())\n", - "pk = random.choice(all_task_pks)\n", - "\n", - "print(EnvironmentProfile.get(pk))" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "colab": { - "base_uri": "https://localhost:8080/", - "height": 1000 - }, - "id": "SFW9PRzzH1Pt", - "outputId": "d53ab793-6740-4e69-9fc7-3a911c892aa1" - }, - "outputs": [], - "source": [ - "# @title Show me an Episode of the random social task before\n", - "\n", - "from sotopia.database import EpisodeLog\n", - "\n", - "all_episodes = list(EpisodeLog.find(EpisodeLog.environment == pk).all())\n", - "if len(all_episodes) == 0:\n", - " print(\"No episodes found for this task, Please try to sample another task.\")\n", - "else:\n", - " episode = random.choice(all_episodes)\n", - " print(episode)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "dYx2N8njZAIi" - }, - "source": [ - "### Find all young chefs in Sotopia\n", - "\n", - "Powered by `redis_om`, the database of sotopia can be easily indexed by simple logics" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 33 - }, - "id": "7mAMGMXgYGRE", - "outputId": "8b85b2fa-c3de-4054-d466-223530ceea18" - }, - "outputs": [], - "source": [ - "young_chefs = AgentProfile.find(\n", - " (AgentProfile.occupation == \"Chef\") & (AgentProfile.age < 45)\n", - ").all()\n", - "print(\n", - " f\"There are {len(young_chefs)} young chefs in sotopia. They are: {', '.join(chef.first_name for chef in young_chefs)}\"\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "8iIG2EWOa1A1" - }, - "source": [ - "### Tagging system in Sotopia\n", - "\n", - "You might notice that Sotopia episodes are all \"tagged\" which allows for tracing the source of the generation. You can list all of the tags" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "sDST-ZcJeubQ", - "outputId": "1750cc27-909d-4b93-9d5d-b3f8e8cfc82d" - }, - "outputs": [], - "source": [ - "def get_episode_tag(episode_id):\n", - " try:\n", - " episode = EpisodeLog.get(episode_id)\n", - " return episode.tag\n", - " except Exception as _:\n", - " return \"\"\n", - "\n", - "\n", - "set(get_episode_tag(episode) for episode in EpisodeLog.all_pks())" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "9YNRGLsRfMvG" - }, - "source": [ - "## Conclusion\n", - "\n", - "In this very short tutorial, I demonstrated how to load the existing data from sotopia(-pi) experiments and browse the data with the easy-to-use api in sotopia. You can enumerate or find a random character and social task, and you can also index the interaction episodes with the task id or character id." - ] - } - ], - "metadata": { - "colab": { - "provenance": [] - }, - "kernelspec": { - "display_name": "Python 3", - "name": "python3" - }, - "language_info": { - "name": "python" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -}