Skip to content

Commit

Permalink
fix links
Browse files Browse the repository at this point in the history
  • Loading branch information
molly-firebolt committed Dec 1, 2023
1 parent 9192730 commit 3a0c107
Show file tree
Hide file tree
Showing 20 changed files with 45 additions and 46 deletions.
14 changes: 7 additions & 7 deletions docs/Guides/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ To start working with data, you first create a database and a *Firebolt engine*.
3. Under **Database engines**, leave the default engine selected. Firebolt will name the engine *Tutorial_general_purpose*.

4. Choose **Create database**.
![](assets/images/newdb.png)
![](../assets/images/newdb.png)
Firebolt adds your database to the **Databases** page.


Expand All @@ -44,7 +44,7 @@ SHOW DATABASES;
```

3. Choose **Run Script** and note that the **Using** list indicates the engine that Firebolt uses to run the script, for example, `Tutorial_general_purpose`.
![](assets/images/showdb.png)
![](../assets/images/showdb.png)

4. When Firebolt prompts you to start the engine, choose **Start Engine**. The engine will take a few minutes to set up.

Expand Down Expand Up @@ -103,7 +103,7 @@ TYPE = (CSV SKIP_HEADER_ROWS = 1);

3. Choose **Run Script**.
Firebolt creates the external table. When finished, the external table `ex_levels` appears on the object panel of the database.
![](assets/images/exlevels.png)
![](../assets/images/exlevels.png)

4. Choose the vertical ellipses next to **Script 2**, choose **Save script**, enter a name (for example, *MyExTableScript*) and then press ENTER to save the script.

Expand Down Expand Up @@ -145,7 +145,7 @@ PRIMARY INDEX LevelID;

3. Choose **Run Script**.
Firebolt creates the fact table. When finished, the table `levels` appears on the object panel of the database.
![](/assets/images/createfacttable.png)
![](../assets/images/createfacttable.png)

### Use INSERT to ingest data
You can now use the `INSERT` command to copy the data from the external table into the fact table. During this operation, Firebolt ingests the data from your source into Firebolt.
Expand Down Expand Up @@ -181,9 +181,9 @@ FROM ex_levels WHERE SOURCE_FILE_TIMESTAMP > (SELECT COALESCE(MAX(SOURCE_FILE_TI
```
3. Choose **Run Script**.
The query results pane indicates a **Status** of **Running** as shown below.
![](assets/images/running.png)
![](../assets/images/running.png)
The **Status** changes to **Success** when the ingestion is complete as shown below.
![](assets/images/success.png)
![](../assets/images/success.png)

## Query the ingested data
Now that the data has been ingested into the `levels` table, you can run analytics queries over the table that benefit from the speed and efficiency of Firebolt.
Expand All @@ -197,7 +197,7 @@ FROM
levels
```
The values shown in the query results pane should be similar to those shown below.
![](assets/images/results.png)
![](../assets/images/results.png)

### Configure an aggregating index

Expand Down
26 changes: 13 additions & 13 deletions docs/Guides/loading-data/incrementally-loading-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,23 +47,23 @@ CREATE FACT TABLE IF NOT EXISTS rankings
) PRIMARY INDEX gameid, playerid;
```

## Set up an Airflow connection to Firebolt
<!-- ## Set up an Airflow connection to Firebolt
To get started connecting Airflow to Firebolt, use the Apache Airflow provider package for Firebolt, `airflow-provider-firebolt`. For more information, including requirements to set up the connection, see [Connecting to Airflow](../integrations/data-orchestration/airflow.md).
To get started connecting Airflow to Firebolt, use the Apache Airflow provider package for Firebolt, `airflow-provider-firebolt`. For more information, including requirements to set up the connection, see [Connecting to Airflow](../integrations/data-orchestration/airflow.md).
## Create and save an INSERT script
An Airflow DAG consists of tasks, and tasks can run SQL in Firebolt. The DAG you create in the next step references the SQL script below, which you save locally as a file. The script uses the `source_file_name` and `source_file_timestamp` metadata virtual columns to determine the records to load from Amazon S3. The `WHERE` clause filters records so that Firebolt loads only those with file timestamps later than any already in the table.
Create a subdirectory of your Airflow home directory with a name of your choosing (for example, `sql_store`). Save the SQL file locally in that directory as `data_load.sql`, with the contents shown below.
Create a subdirectory of your Airflow home directory with a name of your choosing (for example, `sql_store`). Save the SQL file locally in that directory as `data_load.sql`, with the contents shown below.
**Contents of data_load.sql**
```sql
INSERT INTO rankings_information
SELECT
*,
source_file_name,
SELECT
*,
source_file_name,
source_file_timestamp
FROM rankings
WHERE source_file_timestamp > (SELECT MAX(source_file_timestamp) FROM rankings);
Expand All @@ -77,14 +77,14 @@ The task in the DAG uses a Python function and an Airflow variable to refer to t
| :--------------------- | :-------------------------------------------------------------- | :------------ |
| firebolt_sql_path | The subdirectory of Airflow home where your SQL file is saved. | `sql_store` |
| firebolt_default_region | The AWS Region of your database. | `us-east-1` |
For more information, see [Variables](https://airflow.apache.org/docs/apache-airflow/stable/concepts/variables.html) and [Managing Variables](https://airflow.apache.org/docs/apache-airflow/stable/howto/variable.html) in Airflow documentation. In the example, the variable key is `firebolt_sql_path`. Use Airflow to set the value of that variable to your fully qualified subdirectory name. The DAG also has functions to run Firebolt operations.
## Create and save the DAG
The DAG script shown below uses the `schedule_interval` DAG property to run the DAG periodically. The `schedule_interval` can be a cron expression or one of several pre-defined intervals. In this tutorial, we use the cron expression `* * * * *`. This runs the DAG every minute. The DAG defines a task `task__incremental_ingest`. This task runs the SQL script using the parameters defined in the Firebolt connection.
The DAG script shown below uses the `schedule_interval` DAG property to run the DAG periodically. The `schedule_interval` can be a cron expression or one of several pre-defined intervals. In this tutorial, we use the cron expression `* * * * *`. This runs the DAG every minute. The DAG defines a task `task__incremental_ingest`. This task runs the SQL script using the parameters defined in the Firebolt connection.
Create a `dags` subdirectory of your Airflow home directory, and then save the DAG file below in that directory with a `*.py` file extension.
Create a `dags` subdirectory of your Airflow home directory, and then save the DAG file below in that directory with a `*.py` file extension.
```python
import time
Expand Down Expand Up @@ -133,19 +133,19 @@ def get_engine_by_name():
engine = fb_rm.engines.get_by_name(name=FIREBOLT_ENGINE_NAME)
return engine
### Function to read the sql file saved locally.
### Function to read the sql file saved locally.
def get_query(query_file):
return open(query_file, "r").read()
### DAG definition that specifies a per-minute schedule.
### DAG definition that specifies a per-minute schedule.
dag = DAG('firebolt_provider_incremental_ingest',
default_args=default_args,
template_searchpath=tmpl_search_path,
schedule_interval='* * * * *',
catchup=False,
tags=["firebolt"])
### Task definition that runs the SQL script saved in the data_load.sql file.
### Task definition that runs the SQL script saved in the data_load.sql file.
task__incremental_ingest = FireboltOperator (
dag=dag,
task_id="task__incremental_ingest",
Expand All @@ -160,4 +160,4 @@ task__incremental_ingest = FireboltOperator (
Make sure that the Firebolt engine specified in your connection is running.
In the Airflow UI, choose **DAGs**. Find your DAG in the list (`firebolt_provider_incremental_ingest`), choose the play button, and then choose **Trigger DAG**.
![](../assets/images/airflow_trigger_dag.png)
![](../assets/images/airflow_trigger_dag.png) -->
4 changes: 2 additions & 2 deletions docs/Guides/loading-data/working-with-external-tables.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ parent: Load data

Loading data into Firebolt is done using *external tables*. External tables are different from [fact and dimension tables](../../Overview/working-with-tables/working-with-tables.md). Firebolt uses an external table as a connector to your data source. No data is stored in Firebolt.

To create an external table, run the [CREATE EXTERNAL TABLE](../../sql-reference/commands/data-definition/create-external-table.md) command. After you create an external table, use the [INSERT](../../sql-reference/commands/data-management/insert.md) command to load the data from the external table into a fact or dimension table. Data that you ingest must be in the same AWS Region as the target Firebolt database.
To create an external table, run the [CREATE EXTERNAL TABLE](../../sql_reference/commands/data-definition/create-external-table.md) command. After you create an external table, use the [INSERT](../../sql_reference/commands/data-management/insert.md) command to load the data from the external table into a fact or dimension table. Data that you ingest must be in the same AWS Region as the target Firebolt database.

{: .caution}
Although you can run a query over an external table to return query results, we don't recommend it. Such a query will be significantly slower than the same query run over the same data in a fact or dimension table because of the data transfer between Firebolt and your data store. We strongly recommend that you use external tables only for ingestion, specifying the table and its columns only in the `FROM` clause of an `INSERT` statement.
Expand Down Expand Up @@ -39,7 +39,7 @@ The metadata virtual columns listed below are available in external tables.
| `source_file_name` | The full path of the row data's source file in Amazon S3, without the bucket. For example, with a source file of `s3://my_bucket/xyz/year=2018/month=01/part-00001.parquet`, the `source_file_name` is `xyz/year=2018/month=01/part-00001.parquet`. | TEXT |
| `source_file_timestamp` | The UTC creation timestamp in second resolution of the row's source file in S3. | TIMESTAMPTZ |

For examples of metadata virtual column usage, see [Extracting partition values using INSERT](../../sql-reference/commands/data-management/insert.md#extracting-partition-values-using-insert) and [Incrementally loading data with Airflow](incrementally-loading-data.md).
For examples of metadata virtual column usage, see [Extracting partition values using INSERT](../../sql_reference/commands/data-management/insert.md#extracting-partition-values-using-insert) and [Incrementally loading data with Airflow](incrementally-loading-data.md).

### Example&ndash;querying metadata virtual column values

Expand Down
2 changes: 1 addition & 1 deletion docs/Guides/managing-your-organization/managing-logins.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ grand_parent: Guides
# Manage logins
{: .no_toc}

Logins are managed at the organization level and are used for authentication. Logins are a combination of a login name (email), first name, last name, and password, unless you've configured [Single Sign-On (SSO)](sso/sso.md). Moreover, logins can be configured with advanced authentication properties such as [MFA](mfa.md) and [network policies]. Logins are linked to users at the account level, so that roles may be managed separately per account. A user must be linked to either a login or a service account for programmatic use to gain access to Firebolt. You can add, edit or delete logins using SQL or in the UI.
Logins are managed at the organization level and are used for authentication. Logins are a combination of a login name (email), first name, last name, and password, unless you've configured [Single Sign-On (SSO)](sso/sso.md). Moreover, logins can be configured with advanced authentication properties such as [MFA](enabling-mfa.md) and [network policies]. Logins are linked to users at the account level, so that roles may be managed separately per account. A user must be linked to either a login or a service account for programmatic use to gain access to Firebolt. You can add, edit or delete logins using SQL or in the UI.

To view all logins, click **Configure** to open the configure space, then choose **Logins** from the menu, or query the [information_schema.logins](../../sql_reference/information-schema/logins.md) view.

Expand Down
2 changes: 1 addition & 1 deletion docs/Guides/managing-your-organization/network-policies.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Network policies can be configured on the organization level and also per login

The IP allow and blocked lists used to specify a network policy are specified as comma-separated IPv4 addresses and/or IPv4 address ranges in CIDR format. You can apply the same list to one or many users, and each user can have unique lists. You can specify lists manually or import lists of addresses and ranges from a CSV file saved locally. You can add, edit or delete network policies using SQL or in the UI.

To view all network policies, click **Configure** to open the configure space, then choose **Network policies** from the menu, or query the [information_schema.network_policies](../../sql_reference/information-schema/network-policies.md) view.
To view all network policies, click **Configure** to open the configure space, then choose **Network policies** from the menu, or query the [information_schema.network_policies](../../sql_reference/information-schema/network_policies.md) view.

{: .note}
Managing network policies requires the org_admin role.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -120,13 +120,13 @@ where

This corresponds to the following setup in OneLogin, where name/surname in OneLogin corresponds to values in JSON:

<img src="../../assets/images/onelogexample1.png" alt="Edit first name" width="500"/>
<img src="../../../assets/images/onelogexample1.png" alt="Edit first name" width="500"/>

## Salesforce

### Configure Salesforce application

<img src="../../assets/images/salesforce1.png" alt="Salesforce config" width="500"/>
<img src="../../../assets/images/salesforce1.png" alt="Salesforce config" width="500"/>

1. In Salesforce, search for the 'Identity provider' setting. Make sure that 'Identity Provider Setup' is enabled. You can use default certificate.

Expand Down Expand Up @@ -193,7 +193,7 @@ where
- Save the value of Single SignOn Service
6. Open the **Attribute Mappings** tab, and edit the saml_subject to map to Email Address. Then **Save**.

<img src="../../assets/images/pingfed6.png" alt="Pingfed" width="500"/>
<img src="../../../assets/images/pingfed6.png" alt="Pingfed" width="500"/>

### Example - Firebolt organization configuration to work with PingFederate
Values for SQL to create the SSO connection are as follows:
Expand Down
2 changes: 1 addition & 1 deletion docs/Guides/managing-your-organization/sso/sso.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ Once SSO configuration is deleted:

## Log in using SSO

1. Go to [go.firebolt.io/login](go.firebolt.io/login).
1. Go to <a href="https://go.firebolt.io/login">go.firebolt.io/login</a>.
2. Enter the name of your organization and choose **Continue to log in**.
If you can’t remember the name of the organization, next to “Don’t know your organization name?” and choose **Find out**.
3. Enter the email address you use to log in to Firebolt and choose **Send link**.
Expand Down
2 changes: 1 addition & 1 deletion docs/Guides/query-data/using-the-sql-editor.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ Firebolt uses cookies in your client to save the state of unsaved scripts automa
![](../../assets/images/global_script.png)

Firebolt moves the script to the **Global scripts** list.
![](../assets/images/global_script_list.png)
![](../../assets/images/global_script_list.png)

**To rename a script**<a name="scriptrename"></a>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ STOP ENGINE MyDatabase_MyFireboltEngine
```

## To create an engine
* Using a running engine, execute a `CREATE ENGINE` statement similar to one of the examples below. For more information, see [CREATE ENGINE](../../sql-reference/commands/engines/create-engine.md).
* Using a running engine, execute a `CREATE ENGINE` statement similar to one of the examples below. For more information, see [CREATE ENGINE](../../sql_reference/commands/engines/create-engine.md).

### Example &ndash; Create engine using default values

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ If your application favors speed of ingestion over speed of analytics queries, b

## How to define an aggregating index

To create an aggregating index, use the [`CREATE AGGREGATING INDEX`](../../sql-reference/commands/data-definition/create-aggregating-index.md) statement. This statement specifies a fact table, a subset of columns from the table, and a list of the aggregate functions that commonly run over that table. You can create as many aggregating indexes per table as you need. Each aggregating index is associated with a single fact table.
To create an aggregating index, use the [`CREATE AGGREGATING INDEX`](../../sql_reference/commands/data-definition/create-aggregating-index.md) statement. This statement specifies a fact table, a subset of columns from the table, and a list of the aggregate functions that commonly run over that table. You can create as many aggregating indexes per table as you need. Each aggregating index is associated with a single fact table.

The syntax for the `CREATE AGGREGATING INDEX` is shown below.

Expand Down
2 changes: 1 addition & 1 deletion docs/Guides/working-with-indexes/using-primary-indexes.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ The video below explains sparse indexing. Eldad Farkash is the CEO of Firebolt.

## How you create a primary index

To define a primary index, you use the `PRIMARY INDEX` clause within a [`CREATE TABLE`](../../sql-reference/commands/data-definition/create-fact-dimension-table.md) statement. Although they are optional, we strongly recommend them.
To define a primary index, you use the `PRIMARY INDEX` clause within a [`CREATE TABLE`](../../sql_reference/commands/data-definition/create-fact-dimension-table.md) statement. Although they are optional, we strongly recommend them.

The basic syntax of a `PRIMARY INDEX` clause within a `CREATE TABLE` statement is shown in the example below.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,7 @@ SELECT

You might want to transform a nested array structure to a standard tabular format so that you can expose views to BI tools that can't handle Firebolt array syntax, or you might find the tabular format more natural to query using standard SQL idioms. `UNNEST` serves these purposes.

[UNNEST](../../sql-reference/commands/queries/select.md#unnest) is part of the [FROM](../../sql-reference/commands/select.md#from) clause and resembles a [JOIN](../../sql-reference/commands/queries/select.md#join). Given an `ARRAY`-typed column, `UNNEST` unfolds the elements of the array and duplicates all other columns found in the `SELECT` clause for each array element.
[UNNEST](../../sql_reference/commands/queries/select.md#unnest) is part of the [FROM](../../sql_reference/commands/queries/select.md#from) clause and resembles a [JOIN](../../sql_reference/commands/queries/select.md#join). Given an `ARRAY`-typed column, `UNNEST` unfolds the elements of the array and duplicates all other columns found in the `SELECT` clause for each array element.

A single `UNNEST` acts similarly to `JOIN`. You can use a single `UNNEST` command to unnest several arrays if the arrays are the same length.

Expand Down
2 changes: 1 addition & 1 deletion docs/Overview/architecture-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Unlike the multi-tenant services layer, the compute and storage layers in Firebo

## Compute Layer

The compute layer runs [Firebolt engines](../Guides/working-with-engines/). Engines are compute clusters that run database workloads. Each engine is an isolated cluster. Within each cluster, engine nodes store data and indexes in the local cache. The engine loads data from the storage layer into cache at query runtime based on the query configuration.
The compute layer runs [Firebolt engines](../Guides/working-with-engines/working-with-engines.md). Engines are compute clusters that run database workloads. Each engine is an isolated cluster. Within each cluster, engine nodes store data and indexes in the local cache. The engine loads data from the storage layer into cache at query runtime based on the query configuration.

A benefit of the decoupled storage and compute architecture is that multiple engines can be assigned to the same database. This allows for granular control over which hardware is assigned to which tasks. Each engine can have a different configuration and size depending on the workloads. Engines can work in parallel or separately, and you can share them with different people in your organization.

Expand Down
Loading

0 comments on commit 3a0c107

Please sign in to comment.