---
title: "How to use the DataRefiner to ingest a file?"
canonical: "https://onesaitplatform.refined.site/space/DOCT/2220794194/How%20to%20use%20the%20DataRefiner%20to%20ingest%20a%20file%3F"
format: markdown
---
## Introduction

This platform module can be accessed by the ADMINISTRATOR and ANALYTICS roles, from the option 

![image](media://8f93a9a8-ea39-4475-929b-f0faa8c74abf)


## Upload file to DataRefiner

Once I access the module from the main page I will select the option

![image](media://77b264fa-401d-49db-ac4a-30b74895852d)

which will enable me the option to load a file:

![image](media://da6af683-ff6c-4669-add9-2a06c4309ceb)

For the file upload example I will use this:

[https://www.iwight.com/documentlibrary/download/transparency-data-april-2013-csv](https://www.iwight.com/documentlibrary/download/transparency-data-april-2013-csv)

![image](media://fc63b1ab-bc0b-4ca9-a8b6-47e79fe8b7af)

 So I'll select this file as URL and hit Next:

![image](media://4377548b-2b4e-4128-a0ae-fef2064607b2)

## Transforming the loaded data

Once the file is loaded I will find a UI like this, from which I can do some transformations prior to loading the file in my project

![image](media://5d08fdb9-d519-49c5-a617-e4a18e57dca2)

For example I can ignore the first line if it has the headers or discard blank rows:

![image](media://ed647ef5-9f97-4520-928d-25e8c8aad10b)

Or try transforming the columns to their type ... we are not going to do this because we will do it at a later stage.

We will conclude by creating the project:

![image](media://90e10515-5647-40bb-b906-ef5ade74a9d9)

Once the project is created I find this UI:

![image](media://e23c02de-94f2-49e8-a55b-ae5e53fa4bca)

Where I can see the first records of the file / project, in this example we have 11285 records.

We also see that each column has a drop-down menu and that a box appears to the left to be able to go back and see the **Facets**.

If I go to the **Amount **column I can see the options I have: **Facet, Filters, edit cells, columns, transpose, sort, reconcile.**

![image](media://75ff08dc-6b98-4304-be71-7070d8662d6a)

 I'll start by saying that this column is of type numeric in **Edit cells> Common transforms> to number**

![image](media://97c54d11-2c26-47e4-a48d-da33bac1572b)

Now I will see the data formatted as numbers, however this does not mean that they are all numbers,

![image](media://8b3a14ae-a739-47ee-8858-4a140172e393)

now I will apply a Facet by selecting **Facet> Numeric Facet**

![image](media://5b710d27-1807-427c-bb17-f3df1a260060)

In the left part I will see that I have 2244 data that is not numeric:

![image](media://b7381bfd-03b2-4734-a2fd-a2eee2f103eb)

If I just leave the Non-numeric check in my table I will see 10 of these records. As you can see, the problem is that there is a comma to separate the thousands and the tool cannot transform it to a number:

![image](media://64c2cc3c-95f6-4a48-895d-982d989c064d)

So I can transform it with the tool and its **GREL (Google Refine Expression Language)** language. I will go to the Amount column, select** Edit cells> Transform** and in the transformation I will indicate **value.replace (',', '')**

![image](media://1a1af075-bc5d-4f9c-b13f-2c5470528d65)

If I apply the filter and apply **Edit cells> Commons Transforms> To number** I can see since all the data in my column is numeric.

![image](media://eff5ebcf-ef4a-422f-b685-b6e9491ff270)

I can apply other more advanced transformations, such as deriving a column from another applying transformations, for that I will put myself in the **Date **column, I will select **Edit column> Add column **based on this column

In this one I will apply the date format transformation:

**value.toDate ('dd.mm.yy'). toString ('yyyy-mm-dd')**

![image](media://4e66ef71-8905-48c0-91f5-890e76e98c55)

And then I'll apply **Edit cells> Commons Transformations> To date**

![image](media://20932a6f-fede-4adf-98b0-368c0d76d7f3)

I can also combine data, reconcile it, ...

## **Extract transformations**

A very interesting concept is that I can extract the transformations to be able to apply them on other data of the same type (the Excel of the following year in this example), this is done from **Undo-redo / Extract**

![image](media://d3bf5bbc-d3b0-42cd-8da7-b1a59f2f8bb8)

The result is a JSON that I can save:

![image](media://d857dfdd-c32f-4319-a18e-34b60debca68)

And that from the **Apply** option I can paste and apply to other files:

![image](media://602742f6-5659-4236-aa6c-07bc63f7b2fa)

## **Export data**

Finally we will see how to export the data of my project. From the Export option I can export in numerous formats:

![image](media://a58f1d44-e2fd-4193-95ab-91fb68a9c1b1)