diff --git a/source/contributing.md b/source/contributing.md index f4af006..7f512e2 100644 --- a/source/contributing.md +++ b/source/contributing.md @@ -22,9 +22,8 @@ you just want to quickly set up a development environment, please follow the ins ## General Notes 1. The front end currently uses vanilla JS. It would be best to refactor it in a way that is more sustainable for future development. Using some kind of framework (Angular, React, Vue) should help. -2. **Back End** - * The APIs we implemented don't follow the popular RESTful style. We decided to make them structured in a way that's very similar to MAVLink commands/messages. It makes initial development a lot easier but I imagine it will make it difficult to add some more advanced features. I think having RESTful APIs actually does make sense for this project. An example of the resources would be a JSON document of all the telemetry information of the vehicle. The backend should send the correct command according to the current speed in that document, etc. This design should make it easier to implement the front end and makes it more structured. -3. **Security** + +2. **Security** * This is probably the biggest problem of the current implementation. I don't have any experience in this field so I can't provide much information. * Currently, CloudStation cannot be used safely for multiple users. User accounts do not actually link drones to their respective accounts, and there are no safeguards in place to prevent one user from entering the drone ID of a different user. Editing the additional telemetry data displayed in "Other Data" will also affect settings for all users. * You could check who accessed the website by \ @@ -33,7 +32,7 @@ you just want to quickly set up a development environment, please follow the ins `less /var/log/nginx/access.log` \ There maybe many hackers (which is common). -4. **Debug** +3. **Debug** * To reload the server, you could do `bash reload_server.sh`. However, if you updated requirements.txt, you have to do `rm -r ENV` and deploy everything again. * You could use print statements or Python logging, however, it's recommended to use Python logging because it's a good practice. You could configure the logs in settings.py. The logs are not displayed by Django even if you run it in front-end, it's configured by daphne.service. So after you reload the server, you could do `journalctl -u daphne.service | tail` for print statements. You may output it to a log file or use Splunk for it to be more convenient. * Debugger Tools: \ @@ -47,21 +46,18 @@ you just want to quickly set up a development environment, please follow the ins ``` - ## Feature Improvements 1. Enhancing multi-user support and security (see General Notes - Security) * Saving drone IDs for each user * Some sort of safeguard to prevent a user from controlling a drone that is not theirs * Saving telemetry data displayed in "Other Data" individually for each user -2. There is currently no support for takeoff, landing, etc., so vehicle types other than rovers cannot be controlled through CloudStation. -3. The fly-to feature currently sets all fly-to points to 0 altitude, which means it can only be used for rovers. If support for flying vehicles is added, the fly-to feature will need altitude controls. -4. Waypoint missions are currently partially implemented in the backend but have no front-end interface and have not been tested. -5. Controlling multiple drones at a time feels very unintuitive and clunky. Improvements could include: +2. Waypoint missions are currently partially implemented in the backend but have no front-end interface and have not been tested. +3. Controlling multiple drones at a time feels very unintuitive and clunky. Improvements could include: * Automatically selecting the tab of a drone when it is clicked on * Automatically bringing up the popup for a drone when its tab is selected * Automatically selecting the tab of a drone when a pin belonging to that drone is clicked * Color-coding or otherwise differentiating pins belonging to different drones * etc. -6. Adding an HUD (with an artificial horizon, etc.) for the currently selected drone -7. Adding video streaming support for the currently selected drone -8. Using different vehicle icons based on the type of vehicle detected +4. Adding an HUD (with an artificial horizon, etc.) for the currently selected drone +5. Adding video streaming support for the currently selected drone +6. Using different vehicle icons based on the type of vehicle detected diff --git a/source/deployment.md b/source/deployment.md index 5a736ad..9df115d 100644 --- a/source/deployment.md +++ b/source/deployment.md @@ -10,8 +10,6 @@ Before you start, you will need an AWS account (free trial version is OK). The f You will also need a free [MapBox](https://account.mapbox.com/) account. Make sure you can find your Mapbox public token, located on the home page after logging in. #### Deployment Steps -For a step by step video guide, see [here](https://www.youtube.com/watch?v=kNkkfPRBoJo). \ -Another [video](https://youtu.be/MzEE0EAfrfY) for CloudStation Deployment on Amazon Web Services Linux 22.4. 1. Launch an EC2 instance on AWS with Ubuntu 22.4 LTS * t2.micro (free-tier eligible) is good enough to test the deployment. @@ -33,59 +31,22 @@ Another [video](https://youtu.be/MzEE0EAfrfY) for CloudStation Deployment on Ama run ``` cd ~ - git clone https://github.com/CloudStationTeam/cloud_station_deployment.git - bash ~/cloud_station_deployment/setup_server.sh + git clone https://github.com/CloudStationTeam/cloud_station_deployment.git + bash ~/cloud_station_deployment/setup_server.sh YOUR_MAP_BOX_KEY_HERE_NO_QUOTES ``` - This deploys an instance of the CloudStation web server based on the latest commit. Note that the latest commit may be developmental. For a more stable version, you can run setup_server.sh with a flag to indicate a certain release tag, e.g. `bash ~/cloud_station_deployment/setup_server.sh --tag=v3.0`. - * You can see the list of releases [here](https://github.com/CloudStationTeam/cloud_station_web/releases). The tag for the release is indicated by the text following the tag icon to the left of the release icon, e.g. v2.0 or v1.0. + This deploys an instance of the CloudStation web server based on the latest commit. Note that the latest commit may be developmental. For a more stable version, you can run setup_server.sh with a flag to indicate a certain release tag, e.g. `bash ~/cloud_station_deployment/setup_server.sh --tag=v4.0`. + * You can see the list of releases [here](https://github.com/CloudStationTeam/cloud_station_web/releases). The tag for the release is indicated by the text following the tag icon to the left of the release icon, e.g. v4.0. * setup_server.sh does the following： 1. Update Ubuntu - 2. Install NGINX and docker + 2. Install NGINX and docker, Django , configures them 3. Clone cloud_station_web source code - 4. Set up a Python virtual environment (install dependencies) -5. Modify cloud_station_web/webgms/settings.py - * Add EC2 IP address/DNS to ALLOWED_HOST - * DNS example: "ec2-xx-xx-xxx-xxx.us-west-1.compute.amazonaws.com" (it should be a string, please do not forget the quotation marks) - * Set DEBUG to False - * Set MAPBOX_PUBLIC_KEY to your Mapbox public token - -6. Add your Google Map API Public Key to cloud_station_web/.env. \ -Note: no space and no quotes. e.g. GOOGLE_MAP_API_KEY=something. \ -(You have some free credits for free tier by Google Cloud and it does not cost money under $200 free API limit (which is about several dollars per 1k reqs), and you could opt to Not cost money if it does not exceed the free API limit.) - -7. Modify cloud_station_deployment/nginx.conf - * add EC2 IP/DNS address to Line 68: server_name ec2-xx-xx-xxx-xxx.us-west-1.compute.amazonaws.com -8. Configure NGINX, Daphne and Django (run only once) - ``` - sudo usermod -a -G ubuntu www-data #add nginx user to the ubuntu group - # optional - sudo chgrp -R ubuntu * #change file owners to ubuntu group - sudo chmod -R g+r * #change file permissions of file owners - ``` - Then run ```bash ~/cloud_station_deployment/configure_web_server.sh``` - The script does the following: - 1. Write database migrations - 2. Collect staticfiles to ~cloud_station_web/static - 3. Configure NGINX with nginx.conf - 4. Configure systemctl to automatically run Daphne as a service(daphne.service) - 5. Download redis and start running redis in a docker container -10. In your web browser, go to your EC2 instance's DNS address (ec2-xx-xx-xxx-xxx.us-west-1.compute.amazonaws.com) and you should see the CloudStation website. \ -Note that if it didn't work, reload the server by reload_server.sh. - - - - -#### Notes -1. Note: if the CloudStation website link didn't work on Amazon, don't click on it on Amazon but copy and paste it in a new website. -2. Note: public keys for cloud_station_web/.env are all uppercase letters. If you use a lower case (for example, SMT_smt), it won't work. It may work in fg (JS) but not bg (Python). -3. Public keys are in .env not settings.py. \ -Because it's easier to manage if you have other APIs, and it's safer because 1. .env is added in .gitignore and 2. we don't use the APIs in frontend because it's not a secure practice. we used the Mapbox API in frontend because it does not cost money so it doesn't matter. - -4. Note that you have to modify setup_server.sh (and without the tag) and reload_server.sh if your cloud_station_web repo is a different one. - -5. Note that if at some point you did something wrong / you edited requirements.txt, remove all the files and restart the process. \ -Refer to cloud_station_deployment/rmv_env.sh and edit the files. - + 4. Set up a Python virtual environment (install dependencies) + 5. Configures mapbox api key + 6. Write database migrations (Django) + 7. Configure NGINX with nginx.conf + 8. Configure systemctl to automatically run Daphne as a service(daphne.service) + 8. Download redis and start running redis in a docker container +5. In your web browser, go to your EC2 instance's DNS address (ec2-xx-xx-xxx-xxx.us-west-1.compute.amazonaws.com) and you should see the CloudStation website. Note that modern browsers default to https, but for this use http:// in front of the IP address. #### Redeployment To reload the server (after a code update) @@ -97,13 +58,18 @@ To reload the server (after a code update) 4. Reload NGINX and Daphne 5. Run django_background_tasks +Note that If frontend didn't work, for example If JS files are not updated, clear browser cache. (Typically control shift R to reload). +#### Testing Using SITL +To test CloudStation with a simulated drone instead of a real drone, you can install and run [SITL](https://ardupilot.org/dev/docs/sitl-simulator-software-in-the-loop.html). If you have a Windows computer running Cygwin, you can also use our [SITL deployment script](https://github.com/CloudStationTeam/sitl_deployment) to automate running multiple SITL instances. +For the example SITL command, -Note that If frontend didn't work, for example If JS files are not updated, clear browser cache. +cd ~/ardupilot/ArduCopter; sim_vehicle.py --console --map --osd --out=udp:35.94.121.200:14550 --custom-location=33.64586111,-117.84275,25,0 -#### Testing Using SITL -To test CloudStation with a simulated drone instead of a real drone, you can install and run [SITL](https://ardupilot.org/dev/docs/sitl-simulator-software-in-the-loop.html). If you have a Windows computer running Cygwin, you can also use our [SITL deployment script](https://github.com/CloudStationTeam/sitl_deployment) to automate running multiple SITL instances. +the address of the cloudstation instance is 35.94.121.200. You can use your cloudstation instance IP address instead. + +Professor Burke's github has a single script that will deploy SITL to a linux machine, see: https://github.com/PeterJBurke/CreateSITLenv. #### Restarting Ubuntu If you need to restart Ubuntu, run the following: @@ -118,55 +84,12 @@ There are many ways to configure the drone to do this. The easiest is to have a [J70] Peter J. Burke “A Safe, Open Source, 4G Connected Self-Flying Plane With 1 Hour Flight Time and All Up Weight (AUW) <300 g: Towards a New Class of Internet Enabled UAVs” IEEE Access, 7(1), 67833 – 67855 (2019). -## AWS RDS (Aurora engine) - Experimental -Note that the project uses SQLite due to its low cost and ease of use with Django. However, AWS RDS can be configured for scalability and robustness. -1. Launch an RDS instance on AWS with Aurora with MySQL compatibility - * db.r5.large is good enough to test the deployment - * Configure Security Group (AWS EC2 Console) - * MySQL/Aurora(TCP) Port:3306 Source:My RDS endpoint -2. Add the RDS_DB_NAME, RDS_USERNAME, RDS_PASSWORD, RDS_HOSTNAME and RDS_PORT to the environment variables - * You can do this by editing the ~/.env file and adding the variables in the following format. - ``` - variable_name=value - ``` - * Add this to the ~/.bash_profile file - ``` - set -a - . ~/.env - set +a - ``` - - * Make sure to run this command after editing the ~/.bash_profile file. - ``` - source ~/.bash_profile - ``` - -3. Edit the cloud_station_web/webgms/settings.py file and change the DATABASES field to the following - ``` - import os - - DATABASES = { - 'default': { - 'ENGINE': 'django.db.backends.mysql', - 'NAME': os.environ['RDS_DB_NAME'], - 'USER': os.environ['RDS_USERNAME'], - 'PASSWORD': os.environ['RDS_PASSWORD'], - 'HOST': os.environ['RDS_HOSTNAME'], - 'PORT': os.environ['RDS_PORT'], - } - } - ``` -4. Edit the cloud_station_deployment/backgroundtasks.service and add this line to the [Service] section. - ``` - EnvironmentFile=/home/ubuntu/.env - ``` -5. run ```bash ~/cloud_station_deployment/configure_web_server.sh``` -6. run ```bash ~/cloud_station_deployment/reload_server.sh``` ## Authors * Lyuyang Hu * Omkar Pathak + * Peter Burke ## Troubleshooting 1. How do I know my hardware setup is correct (the vehicle is sending mavlink messages to the server)? @@ -176,9 +99,7 @@ Note that the project uses SQLite due to its low cost and ease of use with Djang service nginx status service daphne status ``` -3. How do I know the status of django_background_tasks? - ```service backgroundtasks status``` -4. The telemetry textbox shows that the websocket connection between server and browser has been disconnected. What do I do? +3. The telemetry textbox shows that the websocket connection between server and browser has been disconnected. What do I do? * This usually means Redis fails. Following the Django Channels recommendation, we use Redis as the backing store for the channel layer. We use Docker to run Redis. * To check status of Docker: ```service docker status``` * To show all Docker containers on the machine: ```sudo docker ps -a``` @@ -188,6 +109,4 @@ Note that the project uses SQLite due to its low cost and ease of use with Djang sudo systemctl enable docker sudo docker run -p 6379:6379 -d redis:2.8 ``` -5. If you are using a Python version past 3.6, Python does not like "pkg-resources" anymore. Go to the requirements.txt file and comment out the following line: -'''pkg-resources==0.0.0''' diff --git a/source/sitl.md b/source/sitl.md index 6183ce2..a6cf82d 100644 --- a/source/sitl.md +++ b/source/sitl.md @@ -1,4 +1,7 @@ -# SITL Deployment +# SITL Deployment (Linus) +See https://github.com/PeterJBurke/CreateSITLenv. + +# SITL Deployment (Windows) For SITL deployment ([An Intro Video](https://youtu.be/KfAOsSYtokA)), you may want to start another Windows ec2 ([An Intro Video](https://youtu.be/L0LnWJfgDeM)) and connect to it by rdp, then 1. download https://github.com/ArduPilot/ardupilot/blob/master/Tools/environment_install/install-prereqs-windows.ps1, then right click to run it by ps (then type A) (to download cygwin and mavproxy). diff --git a/source/webapp.md b/source/webapp.md index 45c9460..1970e28 100644 --- a/source/webapp.md +++ b/source/webapp.md @@ -41,3 +41,8 @@ [x] Add customizable telemetry options [] Link customizable telemetry options to specific users + +[x] Waypoints (Cloudstation V 3.0 (Fall 2023)). This was a student project that crashed the drone. So it is on the back burner for now + +[x] Cloudstation 4.0 (Released January 2024). Add support for all flight modes, arm/disarm clearly on UI, support land/takeoff, RTL. +