For the presentation on Friday the 17th, Scott will be completing the guide on replicating our Dataserver on other machines. He will also write a php script to create and initialize the database and tables I will be changing the upload code to work when the GCRS is down (see below). And I will create diagrams for the concepts of the context aware group messaging architechture (Visio?) The general layout of the presentation will be as follows (I hope): The Android Sensing Lib group will present and reach the topic of storing csv files and uploading data. So this is where we jump in. So we'eve turned one of our old laptops into a LAMP server and in the end we want the archival data to be stored in the MYSQL database of our server. So we all the csv files within the data directory of our device over to our dataserver via HTTP POST requests. We've created a php script on the server side which parses the csv files and constructs MYSQL INSERT queries to run - effectively storing all of the sensordata into their respective tables. Each sensor is given its own table to store data in. And thusly, we direct data to their respective tables based on the the name of the csv file. We've named the csv files with this naming scheme: Sensortype_sensorname.csv ex)1_KXTF9 3-axis Accelerometer.csv and for the sensors not given a type (ad not managed) by Android's sensor manager, we simple label it as gps_.csv or bt_.csv etc On the php side, we simply look for the underscore in the file name and then we know which table to insert the data into. Now for the sake of Mobility First, we've integrated UMass' GCRS with our data. So before we actually send out the files, the device contacts the GCRS to either lookup or register a GUID. The GCRS takes in a username and spits out a GUID, so we've decided to use the phone's IMEI/ESN as the username. The phone also contacts our dataserver for its external ip address and we write the phone's IP to the GCRS as well. And of course, this entire architecture must be scalable to accomodate for larger amounts of phones/users, and thus, larger amounts of data. So the role of our data server must be simple to replicate nad reproduce. Here is how one can do so: --------------------------------------------- Annoying: UMass' GCRS has been moved (url change)... GCRS does not work like it used to; and thus, the portion of my code that grabs the GUID and writes the IP does not really work For now, devices will upload an empty GUID if the GCRS cannot be contacted. ------------------------------------------------- Alright, we're on the last week of work here... Final presentations are on August 17th and the presentation to outsiders will be on August 27th. Need to make a poster; however, the stuff we worked on is pretty segmented. A lot of the work I did in the beginning (data collection / organization on the client side) has just been included into Android Sensing Library Group. Where the data goes when the phone sends it out (Server side stuff) is also mostly related to the sensing library. I wish they would lend me like 20 android phones to gather data for my bluetooth localization... but eh... I think Ivan wants me to use ORBIT. (I don't want to use ORBIT) -------------------------------------------------- should make a php/html webpage with forms that can select 'groups' to message some sort of secret message Ok, so the website has a page with a form that dynamically loads table names from the server. -------------------------------------------------- ok.. took a long break with doing the bluetooth localization Implemented algorithms 1 and 2 fom the MIT paper I was trying to code as fast as possible, so the code is horrid. Need to add my own algorithm to do the cluster manipulation since they don't talk about that pseudo code in the paper... Need data regarding bluetooth RSSI though... ----------------------------------------------------- Red Alert One member has went on vacation to Hawaii, and the other has jumped ship. Well, I might be able to get some work done now. ------------------------------------------------------ OK, so user preferences do not save if you force close out of the application (Talked to Kiran...he said just change the GCRS code directly but I wanted access to svn to change it...whatever, I think he gave notice to the UMass guys about the error) ---------------------------------------- Paper on Localization of Sensor Networks (suing robust quads and trilateration) http://rvsn.csail.mit.edu/netloc/sensys04.pdf Cool. Time to implement. (and cite, so I don't get destroyed for plagerism) ------------------------------- ok added RSSI (to my local copy) also, I never fixed the unique discovery thing??? ------------------------------- regarding bluetooth May want to have settings to filter out certain types of devices (laptops, accessories, phones) the mac address can also tell us the manufacturer? and thus the type of device?? Also (via http://stackoverflow.com/questions/3020407/how-to-find-signal-strength-of-connected-bluetooth-devices For bluetooth RSSI, you can read RSSI for connected devices, or perform a bluetooth discovery to check the RSSI for nearby devices. Basically a bluetooth discovery is a broadcast to all stations within range to respond back. As each devices respons back, Android fires off an ACTION_FOUND intent. Within this intent you can getExtra EXTRA_RSSI to obtain the RSSI. Note that not all bluetooth hardware supports RSSI. Maybe create a whole new table for bluetooth Fields: timestamp, MAC id, Sig Strength, device type?, [[our mac id, id, guid, ip]] and in this case there would be many entries per single timestamp and then the number of entries per timestamp indicate the number of devices seen at a particular time(side note... need to properly close application for it to save stuff to the csv file) -------------------------------------------- Other things we could upload to the GCRS ... contexts or identifiers that are more private location, MAC ID, ---------------------------------------------------- 7/9 There is a problem with userpreferences In the HTTPCLient class, they initialize the userpreferences in the main method which is not running when I make a new HTTPClient I had to modify the declaration of userpreferencess to contain initialization private static Preferences userPreferencess = Preferences.userRoot().node(HTTPClient.class.getName());; We need an actual registration of a username in order to generate the keypair So, it seems I have to clear the database (GCRS) by entering demo mode However.. this is stupid... we cant have the database clear every time... but otherwise, the userpreferences never saves a key for use by just looking up the GUID -------------------------------------------------------------------- UTF BOM charater really screws things up (when reading server responses and trying to do string manipulation) =( --------------------------------- Need to test the ip key value pair !! Fix the "array of mac ID's" ... just now they contain commas which screws up the csv reader Should delimit them with something else I guess (unserscore space or something else?) update: changed the schema for bluetooth data.... timestamp|selfMAC|otherMAC|RSSI rather than have a list of mac ID's per cell, we'll have multiple entries this way, we can make more use of queries etc --------------------------------------------- Tried to SVN the GCRS code (done) Check in with jacket group about bluetooth proximity to other "jackets" (people) (whatever, I should just get it done) ----------------------------------------- Note that I had changd the HTTP client such that it would check if GUIDs existed before trying to egister them again.. I guess I would have to do the check in the send code rather than changing the httpclient code ---------------------------------------------------- 7/3 (since when did I start dating these...I guess I'll stop) Put IP address as key value pair on UMass GCRS Naming Scheme... Wifi localization(?) Bluetooth group similarity algorithm File Upload robustness "Install Package" for more servers https://help.ubuntu.com/community/InstallCDCustomization (though, this might be bad since there are a lot of extraneous things bundled in if we do this) May want to make a php file to initialize all the DB/tables The Def(!) ----------------------------------------- upload IP !! ok, done. Instead of uploading, the ip of the uploader is grabbed by the php script which simplifies a ton of stuff ------------------------------------------------- So I have something that generates SQL queries by inputting which sensor, values to compare, and time etc... and you are returned a query... The query, when run in SQl, will give a list of (distinct) GUID's of devices who match the query (obviously) My plan here is to have the human readable context functions call this query generator I plan to have a method that HTTP POSTs the query to a php script which runs the query The method will have certain flags that toggle: create/return a group GUID of the queried devices return JSON array of the guid's ---------------------------------- Bluetooth logger now grabs # of devices and a list of MAC ID's(comma delimited) The list of MAC id's will fill just 1 cell per entry... This is only for the sae of capturing raw data... perhaps for further analysis (seeing how many of the SAME/DIFFERENT people are near you) ------------------------------------- abstract the context info define the context on the phone level...sigh but then the data server is useless? whatever Data server would just be an archival... GCRS server should be abled to be queried Upload IP address as well? Take the clicker app and say theses are there are bunch of these phones guids on each phone monitoring these sensors determine a way of forming the groups for the interest of the applications take one feature of the app and see how we're going to use the GCRS (maybe the dataserver) Role of GCRS????? ------------------------------------------------- Define the contexts on client side and store somewhere -------------------------------------------------- Security of the data Server? (agian) Janne wanted us to implement ssl cert to our data server. I feel like this should be done later, but whatever. Brute force ssl connection done! (but no way am I going to be using it until later) http://blog.crazybob.org/2010/02/android-trusting-ssl-certificates.html http://stackoverflow.com/questions/2642777/trusting-all-certificates-using-httpclient-over-https/6378872#6378872 http://blog.antoine.li/2010/10/22/android-trusting-ssl-certificates/ --------------------------------------------- Need to look into SQLite I don't want to throw away all the work of saving to csv files... but I guess we should store in local SQLite databases as well then Both csv and db files will still allow opportunistic uploading (perhaps an option to choose to write to db or csv, or both in the case where the user without root would like to easily access their sensorData) ------------------------------------------- Security of the data Server? (agian) TLS / SSL HTTPS connection? But neither our server nor the UMASS server have HTTPS (that means they don't support TLS/SSL right now?) ---------------------------------------------- Able to HTTP GET GUID from UMASS server using the phone's IMEI as the registration name The phone's GUID is also sent as an ID parameter ------------------------------------------------ Finally used POST to pass both parameters and files to sql So far the only parameter sent is the phone's IMEI ID -------------------------------------- Hard coded a switch case for column field names The 3-valued sensors are `x` `y` `z` (always) and the singular valued sensors are `value` `null0` `null1` because they return 3 values for some stupid reason Changed the field names in the SQL database accordingly ---------------------------------------- Need to still: Send all files at once(???) [done] Manual send button Option to only send on Wifi Turn logging on or off Set the logging as a service ------------------------------------------------ Additional fields may include GUID, ip, IMEI etc ------------------------------------------------- Changed the schema: timestamp | x | y | z | (additional fields) and timestamp | value| (additional fields) -------------------------------------------------- Manual hard coded csv file upload from phone to SQL is done (Note that the file upload portion of the android client requires permission to access internet In PHP, we use $filename =$_FILES["file"]["name"]; for the html submit form but we have to use $filename =$_FILES['uploadedfile']['name']; when sending the http post from the android client ---------------------------------------- Assigning GUIDS: huh? Java client (Kiran...where you at?) or HTTP post/get Sending the files from the phone: Manual send button Option to only send on Wifi csv file name ---> db table: The number before the underscore "_" in the name "1_KXTF9 3-axis Accelerometer.csv" determines which sensor table the data should fall into. (it would be easier if the tables were just named as numbers, but then they'd be undescriptive of which sensor..) When parsing, stop at the underscore, rather than just pulling the first digit since the numbers hit the double digits. Also, for gps.csv.... what to do... Should we rename it to gps_(provider).csv or something like 101_gps.csv hopefully the sensor types never reach 100... --------------------------------------------------------- guid per each sensor(?) expand the clicker idea to a general classroom helper For now, rather than JSON objects, we log the data onto csv files and then try to send them to a server to parse it with php and push into sql tables ------------------------------------------------------------------ Have a listener to the server Post data when the server requests it (Rather than onSensorChanged) ---------------------------------------------------------- Need to give app ideas. David proposed this grid conquering thing. It was ok, but too much work. I wanted just a simple proof of concept; to just form a context based on location, and then literally send a message to them. The others did not seem to lie the idea. Scott proposed something related to driving...I suggested something that would aid ParkNet. Nah... On the commute to WINLAB, I spoke to Scott about replacing Rutger's iClicker using context aware novelties. He agreed that it would be good because of attendance issues with the iClicker. We presented the idea to our advisors, most of them liked the idea. Dr.Ray was not impressed. Oh well, majority rules. --------------------------------------------------------------------- I want to push out JSON objects from the android client Things to push out: Sensor Name / ID Sensor Event # Value Possible syntax: ID,event#,value,event#,value,event#,value ...etc (\n for each new sensor type) ID,event#,value,event#,value,event#,value ...etc ID,event#,value,event#,value,event#,value ...etc etc... (it seems there could be a hardcoded 3 # of events 0,1,2) Server should parse this accordingly. (comma dilineated and each event# paired to a value) --------------------------------------------------------------------------- Poll and extract sensor data from device and push it onto the data server An external application can look at the the data The app can define groups based on the sensor data and message devices just based on these groups Also, instead of polling for the sensor data (which can be an expensive task and drain battery), we should interrupt and ask for data only when needed (and from only the specific sensor needed) Data/sensors include: -Accelerometer -Orientation -Camera -Microphone/sound -Proximity -Radio (various: WiFi Bluetooth 3G celluar) -GPS -Time Some phones might have: -Temperature -gyroscope -light -hall effect/mag -pressure Things to work on: -Pushing the sensor data onto the data server -Reading the data server -Define a schema for querying the data from an application -Think of cool applications -Security (encyption or privilieges) The schema for the sensor data should probably be Data ID Value Time http://mylifewithandroid.blogspot.com/2010/03/sensors.html http://pallergabor.uw.hu/androidblog/ (that is, regarding the liscense info) http://stackoverflow.com/questions/983761/android-java-simple-send-and-recieve-with-server-fast-setup-challenge http://developerlife.com/tutorials/?p=288 ------------------------------------------------------ We have recieved our project briefing. We will be working on "Context-aware Group Messaging". ------------------------------------------------------ Hello Captian's_Log... this is Jason Tam. I am an undergraduate student studying EE at The Cooper Union. I have decided to partake in a summer internship at WINLAB (part of Rutgers University).