merge current changes

.gitignore

@@ -2,6 +2,7 @@
 __pycache__/
 *_cache.sqlite
 out/
+*.out.png
 .coverage
 .vscode/
 

README.md

@@ -12,6 +12,18 @@ The project's name is Bowser, named after [Mary Bowser](https://en.wikipedia.org
 
 We are focusing on domestic terrorism.
 
+## Presentation
+
+[You can view our final presentation PDF here.](final-presentation.pdf)
+
+## Images
+
+![](.screenshots/screen1.png)
+
+![](.screenshots/screen2.png)
+
+![](.screenshots/screen3.png)
+
 ## Setup
 
 Clone or download this repository.
@@ -44,13 +56,13 @@ Edit the end of `bowser.py` to make larger CSV files or gather from different bo
 
 #### Web UI
 
-`cd client` and run `npm build`.
+`cd client` and run `npm run-script build`.
 
-This generates JS and HTML that will be served under `0.0.0.0:1839/` as Flask static files.
+This generates JS and HTML that will be served under `0.0.0.0:1839/` as Flask static files, when Flask is running.
 
 Alternatively, you can run `npm start` if you wish to run the web UI under Node.js.
 
-#### Web API/Alternate Web UI
+#### Web API/Static web interface
 
 `cd bowser`
 
@@ -60,15 +72,19 @@ This will start an HTTP API on <http://0.0.0.0:1839/api>.
 
 Also, a web UI from Node.js static files will be served on <http://0.0.0.0:1839/>.
 
+This web UI must be built using `npm` as described in the "Web UI" section.
+
 ##### Heroku instance
 
-Alternatively, <http://bowser-webapi.herokuapp.com/api> can be used if you need to access it over the internet and not locally.
+Alternatively, <http://bowser-webapi.herokuapp.com> can be used if you need to access it over the internet and not locally.
 
 The Heroku app may fail due to Cloudflare WAF blocking Heroku's IPs. Running it locally should prevent this.
 
 ##### DigitalOcean instance
 
-<http://bowser-web-app.freemyip.com:1839> can be used with no restrictions to browse a live version of our project. Try going to `/api`!
+<strike><http://bowser-web-app.freemyip.com:1839> can be used with no restrictions to browse a live version of our project. Try going to `/api`!</strike>
+
+The DigitalOcean instance was shut down as I (Henry P) have graduated and do not want to pay for it. Please run it yourself if you want to use this project.
 
 ### The scrapy test
 
@@ -126,19 +142,19 @@ Role: DevOps Engineer
 
 ### Cooper Van Kampen
 
-Role: Machine Learning AI Cloud Malware Devloper (Agile)
+Role: Cloud Developer, Backend Developer
 
 ### Daniel Denekew
 
-Role: Analysts - Big Data Generate Development
+Role: Data Analyst
 
 ### Rawad Alahmadi
 
-Role: Project Assistant, Machine Language Data Scientist
+Role: Assistant Project Manager, Data Analyst
 
 ### Robert Bacius
 
-Role: Cloud AI Data Analysis
+Role: Data Analyst
 
 ## Extra Documentation
 

bowser/bowserHTTPAPI.py

@@ -64,7 +64,9 @@ def index():
 		"route-url": url_for("routes"),
 		"example-urls": [
 			(url_for('generate_csv') + "?boards=x,pol&flaggers=NSA_PRISM,TERRORISM&start_page=3&stop_page=10"),
-			(url_for('generate_json') + "?boards=pol,s4s,x&flaggers=NSA_ECHELON,RACISM&start_page=1&stop_page=3")
+			(url_for(
+				'generate_4plebs_json') + "?boards=pol,s4s,x&flaggers=NSA_ECHELON,RACISM&start_page=1&stop_page=3"),
+			(url_for('generate_reddit_json') + '?subreddit=Sino'),
 		]
 	})
 
@@ -196,7 +198,7 @@ def generate_reddit_json():
 					'thread_list': threadlist})
 
 
-def _generate_csv_string(boards: str, flaggers: str, start_page: str, stop_page: str) -> str:
+def _generate_csv_string_4plebs(boards: str, flaggers: str, start_page: str, stop_page: str) -> str:
 	boardsDesc = "The boards on 4chan you wish to gather from."
 	boards = unpack_http_get_list(boards)
 	boards = parameter_blacklist(boards, 'boards', boardsDesc)
@@ -237,7 +239,7 @@ def generate_csv():
 	start_page = request.args.get('start_page', None)
 	stop_page = request.args.get('stop_page', None)
 
-	csvString = _generate_csv_string(
+	csvString = _generate_csv_string_4plebs(
 		boards=boards,
 		flaggers=flaggers,
 		start_page=start_page,
@@ -254,8 +256,8 @@ def generate_csv():
 
 
 @app.route("/api/generate/4chan/json", methods=['GET'])
-def generate_json():
-	csvString = _generate_csv_string(
+def generate_4plebs_json():
+	csvString = _generate_csv_string_4plebs(
 		boards=request.args.get('boards', None),
 		flaggers=request.args.get('flaggers', None),
 		start_page=request.args.get('start_page', None),

client/.env.development

@@ -1,7 +1,7 @@
 REACT_APP_EXAMPLE="hi from dev"
 REACT_APP_BOWSER_API_URL="http://0.0.0.0:1839/generate/csv?boards=x,pol&flaggers=NSA_PRISM,TERRORISM&start_page=1&stop_page=2"
 REACT_APP_JARRON_API_URL="http://192.168.2.11:1839/generate/csv?boards=x,pol&flaggers=NSA_PRISM,TERRORISM&start_page=1&stop_page=2"
-REACT_APP_BOWSER_API_HOST="http://0.0.0.0"
+REACT_APP_BOWSER_API_HOST="http://bowser-web-app.freemyip.com"
 REACT_APP_BOWSER_API_PORT="1839"
-REACT_APP_JARRON_API_HOST="http://192.168.2.11"
+REACT_APP_JARRON_API_HOST="http://bowser-web-app.freemyip.com"
 REACT_APP_JARRON_API_PORT="1839"

client/.env.production

@@ -1,2 +1,2 @@
-REACT_APP_BOWSER_API_HOST="http://206.189.77.214"
+REACT_APP_BOWSER_API_HOST="http://bowser-web-app.freemyip.com"
 REACT_APP_BOWSER_API_PORT="1839"

client/README.md

@@ -1,3 +1,13 @@
+## Description
+
+This application utilizes:
+
+-   [Material UI](material-ui.com/)
+-   [Formik](https://jaredpalmer.com/formik/)
+-   [Yup](https://github.com/jquense/yup)
+-   [React Chartkick](https://chartkick.com/react)
+-   [Easy Peasy State Management](https://easy-peasy.now.sh)<br/>
+
 This project was bootstrapped with [Create React App](https://github.com/facebook/create-react-app).
 
 ## Available Scripts
@@ -19,10 +29,12 @@ See the section about [running tests](https://facebook.github.io/create-react-ap
 
 ### `npm run build`
 
-Builds the app for production to the `build` folder.<br />
+Builds the app for production to the `build` folder.
+
 It correctly bundles React in production mode and optimizes the build for the best performance.
 
-The build is minified and the filenames include the hashes.<br />
+The build is minified and the filenames include the hashes.
+
 Your app is ready to be deployed!
 
 See the section about [deployment](https://facebook.github.io/create-react-app/docs/deployment) for more information.