Public/LabDataStorageEvaluation
1
0
Fork 0
Code Issues Pull Requests Actions Releases Wiki Activity

feat(config): update .gitignore to exclude output artifacts and add config files

Browse Source 
- ignore all output files except for reports and config directories
- add new .done files for tasks 01-11 to track completion status
- include new configuration files for hardware prices and lab setup
- generate process flow and report documents for tasks 02 and 11
This commit is contained in:
administrator
2026-07-11 23:52:28 -04:00
parent a4c0d258eb
commit af91629f71
41 changed files with 1028 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
out/report/tables/t1_storage_footprint.csv Normal file
View File

@@ -0,0 +1,6 @@
format,measured,vs CSV,archival (compressed),projected @600 GB,projected @1.2 TB,projected @6 TB
CSV,149 MB,1.00x,53 MB,600 GB,1.20 TB,6.00 TB
JSON-LD,311 MB,2.09x,57 MB,1.26 TB,2.51 TB,13 TB
SQLite,123 MB,0.83x,55 MB,498 GB,996 GB,4.98 TB
PostgreSQL,397 MB,2.67x,53 MB,1.60 TB,3.21 TB,16 TB
RDF triplestore,1.74 GB,11.70x,126 MB,7.02 TB,14 TB,70 TB
1 format measured vs CSV archival (compressed) projected @600 GB projected @1.2 TB projected @6 TB
2 CSV 149 MB 1.00x 53 MB 600 GB 1.20 TB 6.00 TB
3 JSON-LD 311 MB 2.09x 57 MB 1.26 TB 2.51 TB 13 TB
4 SQLite 123 MB 0.83x 55 MB 498 GB 996 GB 4.98 TB
5 PostgreSQL 397 MB 2.67x 53 MB 1.60 TB 3.21 TB 16 TB
6 RDF triplestore 1.74 GB 11.70x 126 MB 7.02 TB 14 TB 70 TB

36
out/report/tables/t2_measured_medians.csv Normal file
View File

@@ -0,0 +1,36 @@
format,query,wall time,peak RSS,CPU util,read
CSV,q1,0.06 s,4 MB,0%,0.0 MB
CSV,q2,0.17 s,20 MB,16%,2.5 MB
CSV,q3,1.15 s,21 MB,24%,31.8 MB
CSV,q4,0.71 s,21 MB,22%,16.8 MB
CSV,q5,1.09 s,21 MB,23%,31.8 MB
CSV,q6,0.28 s,20 MB,17%,1.2 MB
CSV,q7,1.27 s,21 MB,23%,32.5 MB
JSON-LD,q1,0.11 s,22 MB,10%,1.4 MB
JSON-LD,q2,1.48 s,24 MB,24%,53.1 MB
JSON-LD,q3,9.94 s,24 MB,25%,297.6 MB
JSON-LD,q4,9.92 s,24 MB,25%,297.8 MB
JSON-LD,q5,9.96 s,24 MB,25%,297.6 MB
JSON-LD,q6,9.79 s,24 MB,24%,297.8 MB
JSON-LD,q7,10 s,24 MB,24%,297.9 MB
SQLite,q1,0.06 s,4 MB,0%,0.0 MB
SQLite,q2,0.06 s,4 MB,0%,0.0 MB
SQLite,q3,0.11 s,4 MB,0%,0.0 MB
SQLite,q4,0.06 s,4 MB,0%,0.0 MB
SQLite,q5,0.77 s,25 MB,23%,117.9 MB
SQLite,q6,0.06 s,4 MB,0%,0.0 MB
SQLite,q7,0.71 s,25 MB,23%,93.0 MB
PostgreSQL,q1,0.22 s,42 MB,18%,4.1 MB
PostgreSQL,q2,0.22 s,42 MB,20%,4.1 MB
PostgreSQL,q3,0.22 s,42 MB,18%,4.1 MB
PostgreSQL,q4,0.22 s,42 MB,18%,4.1 MB
PostgreSQL,q5,0.44 s,42 MB,10%,4.1 MB
PostgreSQL,q6,0.22 s,42 MB,19%,4.1 MB
PostgreSQL,q7,0.33 s,42 MB,12%,4.1 MB
RDF triplestore,q1,0.11 s,32 MB,17%,3.6 MB
RDF triplestore,q2,2.81 s,85 MB,24%,375.6 MB
RDF triplestore,q3,46 s,264 MB,25%,5445.1 MB
RDF triplestore,q4,20 s,168 MB,25%,2694.6 MB
RDF triplestore,q5,60 s,264 MB,25%,5390.5 MB
RDF triplestore,q6,0.17 s,46 MB,16%,15.9 MB
RDF triplestore,q7,41 s,264 MB,25%,5380.5 MB
1 format query wall time peak RSS CPU util read
2 CSV q1 0.06 s 4 MB 0% 0.0 MB
3 CSV q2 0.17 s 20 MB 16% 2.5 MB
4 CSV q3 1.15 s 21 MB 24% 31.8 MB
5 CSV q4 0.71 s 21 MB 22% 16.8 MB
6 CSV q5 1.09 s 21 MB 23% 31.8 MB
7 CSV q6 0.28 s 20 MB 17% 1.2 MB
8 CSV q7 1.27 s 21 MB 23% 32.5 MB
9 JSON-LD q1 0.11 s 22 MB 10% 1.4 MB
10 JSON-LD q2 1.48 s 24 MB 24% 53.1 MB
11 JSON-LD q3 9.94 s 24 MB 25% 297.6 MB
12 JSON-LD q4 9.92 s 24 MB 25% 297.8 MB
13 JSON-LD q5 9.96 s 24 MB 25% 297.6 MB
14 JSON-LD q6 9.79 s 24 MB 24% 297.8 MB
15 JSON-LD q7 10 s 24 MB 24% 297.9 MB
16 SQLite q1 0.06 s 4 MB 0% 0.0 MB
17 SQLite q2 0.06 s 4 MB 0% 0.0 MB
18 SQLite q3 0.11 s 4 MB 0% 0.0 MB
19 SQLite q4 0.06 s 4 MB 0% 0.0 MB
20 SQLite q5 0.77 s 25 MB 23% 117.9 MB
21 SQLite q6 0.06 s 4 MB 0% 0.0 MB
22 SQLite q7 0.71 s 25 MB 23% 93.0 MB
23 PostgreSQL q1 0.22 s 42 MB 18% 4.1 MB
24 PostgreSQL q2 0.22 s 42 MB 20% 4.1 MB
25 PostgreSQL q3 0.22 s 42 MB 18% 4.1 MB
26 PostgreSQL q4 0.22 s 42 MB 18% 4.1 MB
27 PostgreSQL q5 0.44 s 42 MB 10% 4.1 MB
28 PostgreSQL q6 0.22 s 42 MB 19% 4.1 MB
29 PostgreSQL q7 0.33 s 42 MB 12% 4.1 MB
30 RDF triplestore q1 0.11 s 32 MB 17% 3.6 MB
31 RDF triplestore q2 2.81 s 85 MB 24% 375.6 MB
32 RDF triplestore q3 46 s 264 MB 25% 5445.1 MB
33 RDF triplestore q4 20 s 168 MB 25% 2694.6 MB
34 RDF triplestore q5 60 s 264 MB 25% 5390.5 MB
35 RDF triplestore q6 0.17 s 46 MB 16% 15.9 MB
36 RDF triplestore q7 41 s 264 MB 25% 5380.5 MB

36
out/report/tables/t3_projected_600gb.csv Normal file
View File

@@ -0,0 +1,36 @@
format,query,projected wall time,flag
CSV,q1,0.06 s,OK
CSV,q2,7.1 min,OK
CSV,q3,73.5 min,IMPRACTICAL
CSV,q4,44.1 min,OK
CSV,q5,69.6 min,IMPRACTICAL
CSV,q6,14.5 min,OK
CSV,q7,81.6 min,IMPRACTICAL
JSON-LD,q1,0.11 s,OK
JSON-LD,q2,92.0 min,IMPRACTICAL
JSON-LD,q3,11.0 h,IMPRACTICAL
JSON-LD,q4,11.0 h,IMPRACTICAL
JSON-LD,q5,11.0 h,IMPRACTICAL
JSON-LD,q6,10.9 h,IMPRACTICAL
JSON-LD,q7,11.2 h,IMPRACTICAL
SQLite,q1,0.06 s,OK
SQLite,q2,1.27 s,OK
SQLite,q3,5.7 min,OK
SQLite,q4,0.06 s,OK
SQLite,q5,47.5 min,OK
SQLite,q6,5.18 s,OK
SQLite,q7,43.7 min,OK
PostgreSQL,q1,0.22 s,OK
PostgreSQL,q2,0.22 s,OK
PostgreSQL,q3,3.42 s,OK
PostgreSQL,q4,11 s,OK
PostgreSQL,q5,3.7 min,OK
PostgreSQL,q6,4.06 s,OK
PostgreSQL,q7,115 s,OK
RDF triplestore,q1,0.11 s,OK
RDF triplestore,q2,3.0 h,IMPRACTICAL
RDF triplestore,q3,2.1 d,FAIL
RDF triplestore,q4,22.1 h,IMPRACTICAL
RDF triplestore,q5,2.8 d,FAIL
RDF triplestore,q6,5.9 min,OK
RDF triplestore,q7,45.3 h,FAIL
1 format query projected wall time flag
2 CSV q1 0.06 s OK
3 CSV q2 7.1 min OK
4 CSV q3 73.5 min IMPRACTICAL
5 CSV q4 44.1 min OK
6 CSV q5 69.6 min IMPRACTICAL
7 CSV q6 14.5 min OK
8 CSV q7 81.6 min IMPRACTICAL
9 JSON-LD q1 0.11 s OK
10 JSON-LD q2 92.0 min IMPRACTICAL
11 JSON-LD q3 11.0 h IMPRACTICAL
12 JSON-LD q4 11.0 h IMPRACTICAL
13 JSON-LD q5 11.0 h IMPRACTICAL
14 JSON-LD q6 10.9 h IMPRACTICAL
15 JSON-LD q7 11.2 h IMPRACTICAL
16 SQLite q1 0.06 s OK
17 SQLite q2 1.27 s OK
18 SQLite q3 5.7 min OK
19 SQLite q4 0.06 s OK
20 SQLite q5 47.5 min OK
21 SQLite q6 5.18 s OK
22 SQLite q7 43.7 min OK
23 PostgreSQL q1 0.22 s OK
24 PostgreSQL q2 0.22 s OK
25 PostgreSQL q3 3.42 s OK
26 PostgreSQL q4 11 s OK
27 PostgreSQL q5 3.7 min OK
28 PostgreSQL q6 4.06 s OK
29 PostgreSQL q7 115 s OK
30 RDF triplestore q1 0.11 s OK
31 RDF triplestore q2 3.0 h IMPRACTICAL
32 RDF triplestore q3 2.1 d FAIL
33 RDF triplestore q4 22.1 h IMPRACTICAL
34 RDF triplestore q5 2.8 d FAIL
35 RDF triplestore q6 5.9 min OK
36 RDF triplestore q7 45.3 h FAIL

6
out/report/tables/t4_hardware_600gb.csv Normal file
View File

@@ -0,0 +1,6 @@
format,configuration,chassis,RAM,disk,extra nodes,total
CSV,1 x 1U Supermicro AS-1015CS-TNR (16 cores); 1 x 64 GB DDR5 ECC RDIMM; 1 x 7.68 TB NVMe U.2,"$4,618","$1,887","$3,995",$0,"$10,500"
JSON-LD,1 x 1U Supermicro AS-1015CS-TNR (16 cores); 1 x 64 GB DDR5 ECC RDIMM; 1 x 7.68 TB NVMe U.2,"$4,618","$1,887","$3,995",$0,"$10,500"
SQLite,1 x 1U Supermicro AS-1015CS-TNR (16 cores); 1 x 64 GB DDR5 ECC RDIMM; 1 x 7.68 TB NVMe U.2,"$4,618","$1,887","$3,995",$0,"$10,500"
PostgreSQL,1 x 1U Supermicro AS-1015CS-TNR (16 cores); 1 x 64 GB DDR5 ECC RDIMM; 1 x 7.68 TB NVMe U.2,"$4,618","$1,887","$3,995",$0,"$10,500"
RDF triplestore,2 x 1U Supermicro AS-1015CS-TNR (16 cores); 22 x 64 GB DDR5 ECC RDIMM; 2 x 7.68 TB NVMe U.2,"$4,618","$41,514","$7,990","$4,618","$58,740"
1 format configuration chassis RAM disk extra nodes total
2 CSV 1 x 1U Supermicro AS-1015CS-TNR (16 cores); 1 x 64 GB DDR5 ECC RDIMM; 1 x 7.68 TB NVMe U.2 $4,618 $1,887 $3,995 $0 $10,500
3 JSON-LD 1 x 1U Supermicro AS-1015CS-TNR (16 cores); 1 x 64 GB DDR5 ECC RDIMM; 1 x 7.68 TB NVMe U.2 $4,618 $1,887 $3,995 $0 $10,500
4 SQLite 1 x 1U Supermicro AS-1015CS-TNR (16 cores); 1 x 64 GB DDR5 ECC RDIMM; 1 x 7.68 TB NVMe U.2 $4,618 $1,887 $3,995 $0 $10,500
5 PostgreSQL 1 x 1U Supermicro AS-1015CS-TNR (16 cores); 1 x 64 GB DDR5 ECC RDIMM; 1 x 7.68 TB NVMe U.2 $4,618 $1,887 $3,995 $0 $10,500
6 RDF triplestore 2 x 1U Supermicro AS-1015CS-TNR (16 cores); 22 x 64 GB DDR5 ECC RDIMM; 2 x 7.68 TB NVMe U.2 $4,618 $41,514 $7,990 $4,618 $58,740

11
out/report/tables/t5_weighted_scores.csv Normal file
View File

@@ -0,0 +1,11 @@
scale,format,search_pts,ram_pts,disk_pts,total
measured,CSV,15.5,20.0,8.3,43.8
measured,JSON-LD,1.7,17.2,4.0,22.9
measured,SQLite,50.0,16.9,10.0,76.9
measured,PostgreSQL,26.2,10.0,3.1,39.3
measured,RDF triplestore,1.3,1.6,0.7,3.6
projected_600gb,CSV,0.6,20.0,8.3,28.9
projected_600gb,JSON-LD,0.1,20.0,4.0,24.0
projected_600gb,SQLite,21.8,15.2,10.0,47.0
projected_600gb,PostgreSQL,50.0,5.5,3.1,58.6
projected_600gb,RDF triplestore,0.0,0.2,0.7,0.9
1 scale format search_pts ram_pts disk_pts total
2 measured CSV 15.5 20.0 8.3 43.8
3 measured JSON-LD 1.7 17.2 4.0 22.9
4 measured SQLite 50.0 16.9 10.0 76.9
5 measured PostgreSQL 26.2 10.0 3.1 39.3
6 measured RDF triplestore 1.3 1.6 0.7 3.6
7 projected_600gb CSV 0.6 20.0 8.3 28.9
8 projected_600gb JSON-LD 0.1 20.0 4.0 24.0
9 projected_600gb SQLite 21.8 15.2 10.0 47.0
10 projected_600gb PostgreSQL 50.0 5.5 3.1 58.6
11 projected_600gb RDF triplestore 0.0 0.2 0.7 0.9