scripts for data fusion
Anaconda with python 3 is recommanded for running python scripts
To dump the data, please use valentino and the 'export to csv' option. The order of the columns should be the same as that in the database.
runing order:
python:
main.py
matlab:
importcsv_and_7
importcsv_ios_7
upsample_and_8
upsample_ios_8
gpsExtracion_main_9
steps before running main.py
1, dump location data from database as location.csv
2, dump activity data from database as activity.csv
3, dump wifi data from database as wifi.csv
4, dump phq9 data from database as phq9.csv
5, dump user mapping data (depressed) from database as mapping.csv
to split big csv file into small files based on uid
'1) split_activity.py'
'1) split_location.py'
'1) split_wifi.py'
input: activity.csv, location.csv, wifi.csv
output: uid*.csv in sub-folders
location: android/uid*.csv & ios/uid*.csv
wifi: ios_wifiraw/uid*.csv & android_wifiraw/uid*.csv
actiity:
deduplication
'2) removedup.py'
input: uid*.csv with duplications
output: uid*.csv w/o duplications
location: ios_remdup/uid*.csv & android_remdup/uid*.csv
wifi: ios_wifiraw_remdup/uid*.csv & android_wifiraw_remdup/uid*.csv
look for a corresponding latitude and longitude match in location data at a given timestamp for a given BSSID in wifi data.
'3) mapgpsdata.py'
input: location/uid*.csv, wifi/uid*.csv
output: matchedgps_androidcheck.csv, matchedgps_ioscheck.csv
merge mapping files from ios and android
'4) mergecsv.py'
input: matchedgps_androidcheck.csv, matchedgps_ioscheck.csv
output: matchedgps_final.csv
cauculate the GPS location for BSSID
'5) calculatemedian_latlong.py'
input: matchedgps_final.csv
output: bssidmappingfile.csv
replace the BSSID in the raw wifi file with coordinates
'6) processwifi.py'
input: bssidmappingfile.csv, wifi/uid*.csv
output: wifiprocess/uid*.csv
data fusion - merge data from diffrernt sources
importcsv_and_7.m
importcsv_ios_7.m
input: wifiprocess/uid*.csv, location/uid*.csv, activity/uid*.csv
output: gps_feature_and.mat, gps_feature_ios.mat
data fusion - upsampling
upsample_and_8.m
upsample_ios_8.m
input: gps_feature_and.mat, gps_feature_ios.mat
output: gps_feature_and_up.mat, gps_feature_ios_up.mat
feature generation
gpsExtracion_main_9.m
feature_extract.m
input: gps_feature_and_up.mat, gps_feature_ios_up.mat
output: dSet_and.mat, dSet_ios.mat
tool scripts, don't need to take care
dbscan.m
gridLegend.m
lldistkm.m