DATA PROCESS - OIC-StatCom

DATA PROCESS - OIC-StatCom

TURKISH STATISTICAL INSTITUTE DATA PROCESS and Data Warehouse Projects in TURKSTAT 31.03.2014 (Muscat, Oman) INFORMATION TECHNOLOGIES DEPARTMENT 1 TURKISH STATISTICAL INSTITUTE

Information Technologies Department Information Technologies Department Harzemli Software Development Group Management Software Group IT Support Group

INFORMATION TECHNOLOGIES DEPARTMENT Network and System Management Group Data Management Group 2 TURKISH STATISTICAL INSTITUTE TurkStat Data Architecture INFORMATION TECHNOLOGIES DEPARTMENT

3 TURKISH STATISTICAL INSTITUTE TurkStat Databases Production Database : Current data is stored and always transaction in it. Institutional Database (Enterprise Database): The main repository of TurkStats harmonized historical data. Dissemination Database : Aggregated time series data created from TurkStat institutional databases. Up to now there are 62 institutional databases 59 dissemination databases

Oracle DBMS is used for all database INFORMATION TECHNOLOGIES DEPARTMENT 4 TURKISH STATISTICAL INSTITUTE Production Databases Mainly data entry systems

Database for web applications OLTP(Online Transactional Processing) Surveys data are stored in this database Huge amount of data entry and frequent data manipulation(inserts, updates, deletes)

Various but not costly SQLs Supports thousands of concurrent users We developed a new generic system (Harzemli) that produces surveys generic using DDL. Regional Offices use this program on the netbooks. INFORMATION TECHNOLOGIES DEPARTMENT

5 TURKISH STATISTICAL INSTITUTE Institutional (Enterprise) Databases Institutional database has analysed, harmonized and controlled consistency eventual time series data. Only for internal users. User roles are defined for the statistical units on the newly created institutional databases. The statistical units can access only their own data using SAS codes , and just have select right. They can never delete, update, insert data by themselves. It is necessity for all departments to work in coordination with IT Department during the creation process of institutional databases.

INFORMATION TECHNOLOGIES DEPARTMENT 6 TURKISH STATISTICAL INSTITUTE Dissemination Databases Dissemination databases are the summary aggregated data repository which are created from institutional databases according to the information request The dissemination database is served on the TurkStat web page They are READ-ONLY system For external users, only confidental data is available It is free of charge, shown on web page

Since it is open to whole world, supports many concurrent users INFORMATION TECHNOLOGIES DEPARTMENT 7 TURKISH STATISTICAL INSTITUTE There are aggregated time-series data about 62 subjects. End users can easily get desired data with the help of interface programs and dynamic reports The current web-based programmes are developed by our programmers with ZK framework and Oracle reports We are developing a new Central Dissemination Project that will show all dissemination data by one software programme

INFORMATION TECHNOLOGIES DEPARTMENT 8 TURKISH STATISTICAL INSTITUTE The main benefits of distinction to these 3 database is: To maintain different business processes on different servers. An advantage is performance. While we are making a bulk insertings or queries on instutional database, dissemination database doesnt effect these procedures. So the performance would not be decreased. Another advantage is only aggregated data are served to internet users

and access to micro data is prevented INFORMATION TECHNOLOGIES DEPARTMENT 9 TURKISH STATISTICAL INSTITUTE Data Harmonization The main subject of the Institutional databases is harmonization of different data After carefull investigations and analyses, it has been seen that our main problems comes from using different classifications from year to year and projects to projects Such as

Branch of economic activity Occupational group Educational status Marital Status Administrative Structure Dictionary tables INFORMATION TECHNOLOGIES DEPARTMENT 10 TURKISH STATISTICAL INSTITUTE Data Warehouse in Turkstat Being holded in our relational institutional database

Designed as star schema model ROLAP (Relational OLAP) which means that all OLAP cubes are on relational database INFORMATION TECHNOLOGIES DEPARTMENT 11 TURKISH STATISTICAL INSTITUTE Data Warehouse in Turkstat (Contd) Being holded in our institution at the scope of European Union projects, Creating data warehouse process was started in 2006 , with SAS And a prototype data warehouse consists of 1985, 1990 and 2000 population censuses were formed

Oracle BI suite is used. INFORMATION TECHNOLOGIES DEPARTMENT 12 TURKISH STATISTICAL INSTITUTE Data Warehouse in Turkstat (Contd) 8 data warehouses / OLAP projects with 48 data marts: 2000 Population Census 1990 Population Census 1985 Population Census Labour Force Statistics Municipal Waste Statistics

Municipal Water Statistics Foreign Trade Statisics Demographical Statistical Datawarehouses in Institutional database. There are different schemas for each Datawarehouse. SQL Procesures are used as ETL, to producing DW data. INFORMATION TECHNOLOGIES DEPARTMENT 13 TURKISH STATISTICAL INSTITUTE Oracle BI Suite

OBIEE 11g is includes : OBIEE Administration Tool OBIEE Interface BI Publisher INFORMATION TECHNOLOGIES DEPARTMENT 14 TURKISH STATISTICAL INSTITUTE OBIEE Administration Tool INFORMATION TECHNOLOGIES DEPARTMENT

15 TURKISH STATISTICAL INSTITUTE OBIEE Interface INFORMATION TECHNOLOGIES DEPARTMENT 16 TURKISH STATISTICAL INSTITUTE OBIEE Interface INFORMATION TECHNOLOGIES DEPARTMENT

17 TURKISH STATISTICAL INSTITUTE OBIEE Interface INFORMATION TECHNOLOGIES DEPARTMENT 18 TURKISH STATISTICAL INSTITUTE OBIEE Interface (pivot table)

INFORMATION TECHNOLOGIES DEPARTMENT 19 TURKISH STATISTICAL INSTITUTE OBIEE Interface (table and chart) INFORMATION TECHNOLOGIES DEPARTMENT 20 TURKISH STATISTICAL INSTITUTE OBIEE Interactive Dashboards

Simply containers for reports and other content (like chart, maps..) Allows multiple reports to be displayed in a tabbed interface Used to provide highly summarized and graphical content INFORMATION TECHNOLOGIES DEPARTMENT 21 TURKISH STATISTICAL INSTITUTE An Example Interactive Dashboard INFORMATION TECHNOLOGIES DEPARTMENT

22 TURKISH STATISTICAL INSTITUTE Summary Databases of the TurkStat are Production, Institutional and Dissemination Seperating databases to 3 database is useful both performance, management, confidentality We have some Data Warehose Projects developed by Oracle BI Suite Oracle BI suite has Administarion Console, Presentetion Interface, Reports Building tool Dashboards are containers that can include various and dynamic reports, maps, charts etc., and give very easy and valuable data to the users INFORMATION TECHNOLOGIES DEPARTMENT

23

Recently Viewed Presentations

  • Harry Potter - WordPress.com

    Harry Potter - WordPress.com

    Pop culture is the hallmark of civilization- advanced societies. Hallmark of capitalistic societies. Society and culture are complex enough and stratified enough. Culture can be commodified- objectified, bought and sold. Theories of Adorno and Horkheimer- criticized/less accepted. Culture is art...
  • Teacher Computer Workshop - Columbus State University

    Teacher Computer Workshop - Columbus State University

    Teacher Computing Workshop Wayne Summers ... State University * Solutions / Resources THE HOUSE STUDENT APP CHALLENGE FIRST LEGO LEAGUE Hackathon Activ8 Summer Camps June 10, 2014 Columbus State University * TSYS School of Computer Science 8:42 min * TSYS...
  • Microbiology - Chapter 3

    Microbiology - Chapter 3

    Microbiology Chapter 3, part 2 The best light microscopes can resolve objects to only about 0.2 - 0.5 microns. It is a function of the energy of visible light and its wavelength (we make really good lenses). To increase resolving...
  • The Forces of Weathering and Erosion

    The Forces of Weathering and Erosion

    In this picture the running water eroded the sides of this canyon and created the V-shaped valleys. Ice Erosion Glaciers This is a picture of a glacier which carves out a U-shaped valley where it flows dragging rocks and boulders...
  • Year 10, Pathway C 2012 New Generations Outcomes

    Year 10, Pathway C 2012 New Generations Outcomes

    Turner's syndrome. Only one sex chromosome (X) Short stature. Webbed neck. Klinefelter's syndrome affects 1 in 500 males, 1 in 1000 females (triplo X) but often is undiagnosed due to little to no visible symptoms. Turner's syndrome affects 1 in...
  • The Importance of Multi-tiered Systems to Improve the School ...

    The Importance of Multi-tiered Systems to Improve the School ...

    The Importance of Multi-tiered Systems to Improve the School Environment and. Support Emotional and Social Development. Susan Barrett, M.A., PBIS National TA Center Michael Lombardo, M.A. Executive Director, ... A Structure and process for education and mental health systems to...
  • Business Process Execution Language for Web Services

    Business Process Execution Language for Web Services

    Business Process Execution Language for Web Services (BPEL4WS)
  • Problem: An Efficient Projection for L1- Regularization Efficient

    Problem: An Efficient Projection for L1- Regularization Efficient

    l. 2. and . l. 1. regularization. Joint Regularization Penalty. How do we penalize solutions that use too many features? Coefficients for. feature 2. Coefficients for. task 2. An l. 1. norm on the maximum absolute values of the coefficients...