Reading up on JUnit and listening to Hank Williams.
Tuesday, July 7, 2009
Monday, July 6, 2009
GSoC: Hackystat, June 30-July 6
Last Week:
The database ended up in a much different (and significantly pared down) form than I had anticipated. Instead of the multitude of node wrappers that I had last week (one for each kind of object in the database), now there is only one kind, and nodes are distinguished between based on relationships. For instance, all "People" nodes are connected to the "People" subreference node. For clarity, here is a picture:
Subreference nodes are in blue, the reference node (database entry point) is green. Relationships are labeled in all caps. This is mostly an implementation issue, but I thought I would touch on it. Having lots of different node types (Plain Old Java Objects wrapping the underlying nodes) is only useful if they are each storing lots of different information. Because as planned, mine weren't (I brought most of the properties out to make them into nodes), it made more sense to kill the proliferation of wrapper nodes. It certainly cleaned up the implementation a lot.
All right, so that is up at http://code.google.com/p/hackystat-analysis-socnet/
It's shiny and commented, too.
Also this week, the Server work began. I watched the screencast on the design of the sensorbase, and the one on building it from sources...
I built the Sensorbase from source this week. This was actually very easy and pleasant. (I was pleasantly surprised!) Yay, Ivy integration! There was an issue with JavaMail and JAXB, but that was ironed out very easily. JavaMail only comes with Java 6 Enterprise Edition, or something like that.
I edited the SensorBase code so that when you ping it, it responds with a Hello World. This did not require much change, actually--I just removed a lot of code from the Server class, added a HelloPing resource, and borrowed code from the Client class to test and make sure it was working. I will put the changed code up on my google project page, though it's not terribly exciting.
Now, Philip says that developing an API really informs the design of a server, so I have the first draft of that. I based it somewhat off the Sensorbase API. I would very much appreciate feedback on the API. I think I hit on most of what was needed, but could totally have left things out. This will go in my google project's wiki as soon as I learn wiki markup for tables. Currently, it's in the downloads section.
| GET | {host}/nodetypes | Returns a list of all of the node types in the graph |
| GET | {host}/nodetypes/{nodetypename} | Returns a representation of the named node type |
| PUT | {host}/nodetypes/{nodetypename} | Create a representation of the named node type (admin) |
| DELETE | {host}/nodetypes/{nodetypename} | Delete the named node type (admin) |
| GET | {host}/relationshiptypes | Returns a list of all the relationship types in the graph |
| GET | {host}/relationshiptypes/{relationshiptypename} | Returns a representation of the named relationship |
| PUT | {host}/relationshiptypes/{relationshiptypename} | Create a representation of the named relationship type (admin) |
| DELETE | {host}/relationshiptypes/{relationshiptypename} | Delete the named relationship type (admin) |
| GET | {host}/clients | Returns a list of all of the clients using the server |
| PUT | {host}/clients/{client} | Returns a representation of the named client |
| POST | {host}/clients/{client} | Updates the representation of the named client |
| DELETE | {host}/clients/{client} | Deletes the named client |
| GET | {host}/nodes | Returns a list of all the nodes in the graph |
| GET | {host}/nodes/{nodetype} | Returns a list of all the nodes of that type in the graph |
| GET | {host}/nodes/{node}/ | Returns a list of all of the nodes connected to the named node |
| GET | {host}/nodes/{node}/{relationshiptype}/{relationshipdirection} | Returns a list of all of the nodes connected to that node by the specified relationship and relationship direction |
| GET | {host}/nodes/{node}/{nodetype} | Returns a list of all of the nodes of the specified type that are connected to the named node |
| GET | {host}/nodes/{node}/{relationshiptype}/{relationshipdirection}/nodes?startTime={tstamp}&endTime={tstamp} | Returns a list of all nodes that were connected to the named node by the specified relationship and relationship direction during the specified time period |
| GET | {host}/nodes/{node}/{nodetype}/nodes?startTime={tstamp}&endTime={tstamp} | Returns a list of all of the nodes of the specified type that were connected to the named node by a relationship between the start time and the end time |
| GET | {host}/nodes/{node}/relationships?startTime={tstamp}&endTime={tstamp} | Returns a list of all of the relationships that were connected to the named node during the specified time interval |
| PUT | {host}/nodes/{node} | Creates a representation of the named node |
| POST | {host}/nodes/{node} | Updates the representation of the named node |
| DELETE | {host}/nodes/{node} | Delete the named node |
| GET | {host}/people | Returns a list of all of the people nodes in the graph |
| GET | {host}/people/users | Returns a list of all of the people nodes in the graph who are users of the system (ie, those who have added the facebook/twitter apps and are submitting data |
| GET | {host}/people/nonusers | Returns a list of all of the people nodes in the graph who are not users of the system (ie, those that are, for instance, friends of users, but are not users themselves.) |
| GET | {host}/people/users/{user} | Returns a representation of the named user |
| PUT | {host}/people/users/{user} | Creates a representation of the named user (admin) |
| POST | {host}/people/users/{user} | Updates the representation of the named user |
| DELETE | {host}/people/users/{user} | Deletes the named user |
| GET | {host}/people/nonusers/{nonuser} | Returns a representation of the named nonuser |
| PUT | {host}/people/nonusers/{nonuser} | Creates a representation of the named nonuser (admin) |
| POST | {host}/people/nonusers/{nonuser} | Updates the representation of the named nonuser |
| DELETE | {host}/people/nonusers/{nonuser} | Deletes the named nonuser |
This Coming Week:
The SocNet Server is top priority. I will be modeling it after the Sensorbase, with a couple of changes. I intend for resources to be loaded from a config file, for one.
Authentication is somewhat of an issue: each client sending data will have a username and password, but what about getting data? There is no simple way for me to ensure that a user who has added the facebook app is the same one who is requesting data. So that is currently an issue that I am not sure how to resolve.
To get a better idea of what I am talking about, here is another picture:

The issue is that there is (currently) no overarching user registration plan. Perhaps such a thing should be added.
Posted by Rachel Shadoan at 8:15 PM 0 comments
Sunday, July 5, 2009
Am writing a REST API specification for the socnet server.
Posted by Rachel Shadoan at 3:30 PM 0 comments
Labels: bananas
Saturday, July 4, 2009
GSOC 2009: Hackystat and the Hello World Server Ping, and other adventures in Servers
Today I made the Sensorbase server code respond to a ping with "Hello, World!"
Basically, this just required adding another resource, in this case, HelloPingResource. I modeled it fairly closely after the PingResource, but I didn't use the client to check to make sure that it had worked. I swiped code from the SensorBaseClient isHost() method to both perform the ping and receive the response, but put it in a separate test class for the HelloPing. I also wrote a JUnit test (my first!), which ran (I think) when I ran ant -f junit.build.xml, but there was so much junk in that output that I couldn't find my little message in all of it.
I've also got the beginnings of an API framework for this server critter that I am building.
Posted by Rachel Shadoan at 3:21 PM 0 comments
Friday, July 3, 2009
Built the sensorbase from source on three different computers today. The only difficulties I ran into are that in some versions of ubuntu, the installed java is not Java Enterprise edition, and therefore doesn't include javx or javamail. However, those things were not difficult to remedy. I have also found the part of the server that responds to a ping!
Posted by Rachel Shadoan at 8:10 PM 0 comments
Labels: bananas
Tuesday, June 30, 2009
GSOC: Hackystat June 22-30
A slightly belated blog entry, for those of you who were waiting with bated breath.
This Previous Week:
Was not as productive as I hoped. I had taken a week off in early June to study like a madwoman for a Chemistry CLEP test, but after failing the practice test hardcore, decided that a little more time was necessary so that I could actually graduate. So, much of last week was spent living like a hobo, perched on the midden heap that was a corner of my sofa, reading the entire chemistry textbook from MIT. Fortunately, I passed, so now I get to graduate.
Here's what I did get squeezed in:
This is mostly finished! I think I redesigned it at least six times, but at the very least, the bones are there. The big thing left to do for it is make an easy way to retrieve information, but since this supposedly is what Neo shines at (traversals and so forth), I think we should be good.
Later this evening I will post a diagram of the design. I decided to go the way of making everything possible a node instead of an attribute. Potentially, this will come back to chew on my butt, but for the time being it made for some really beautifully simple implementation.
My one concern with that path is that I ended up with quite a few classes that did not have much substance. All of the classes extend the abstract SocialMediaNode class, which houses the underlying node and has accessors and mutators for the only property I am requiring all nodes to have, that is, a name. Some of my classes (for instance, the Coder class), have quite a few more properties than just a name. However, there are probably an equal number (perhaps slightly more) that are just sort of wrappers for the abstract superclass, consisting only of a constructor that passes a value to the superconstructor. I did this so it would be easier to distinguish between types of nodes and so the database creation code would be more straightforward, but I do not know that it was the correct decision.
Yeah, I know I said that I wasn't going to work on this this week. But, after a long discussion with Zack about the server issue, I decided that it the Facebook application was going to require the most specific things from the server (it's actually the only thing that needs to be hosted) and therefore be the most pressing.
Now, my original plan was to build the app on using Joyent as my host and then transfer it to another host later, but this is not going to work. Joyent does not provide free hosting for Facebook applications in Java, only Facebook applications in PHP and (I think) Ruby. Now, I have written in Ruby, but it's been almost three years and at this point in the summer I think it would be
So, I started setting up one of Zack's desktops as a host. So, I registered with DynDNS, and installed a dyndns updater. Glassfish is installed and configured. Whee.
The structure of this whole thing changed again. At last update, I was planning to host the application that let users release to me for mining their Hackystat data. However, I decided that it was probably most secure to make that application something they download to their own computer, as then it can access sensorshell.properties and I don't have to securely store authentication information. This also makes this application significantly simpler to write, which is pleasing.
Next Week:
I am a trifle behind (I think I had planned to have the database ready to roll by today), but it should be ready in the bat of an eye span. (I would love to say tomorrow, but if I say tomorrow, overnight I will decide that I need to completely refashion it again and that would automatically kill the database contentedness.) Besides, after last week I've forgotten what fun and leisure are like, so I forsee just oodles of productivity.
- Okay, so database.
- Twitter application. Do we still want this to be sending updates to the sensorbase?
- After those two, the downloadable "Let Rachel Access Your Hackystat Data for Mining" application.
- Also, database commenting. Right now it is naked, and it isn't super hard to understand, but then again, I just wrote it. So commenting it before I forget what I was thinking is a good idea.
- And the server stuff. Philip was so kind as to make a number of screencasts for me, and so I really ought to watch them.
- I suppose I should also actually upload my code to my very lonely Google project site.
I admit that I am a little concerned about where this whole thing is going. My original plan was to have stuff collecting data by now, so that I could do the data mining of it for a graduate independent study in July.
However, I'm not sure that was ever a reasonable goal, particularly considering you know, wanting to really test things before I unleash them on the computers of unsuspecting users. While I feel that I can certainly have everything in place, all of the components built, by the end of the summer, I'm not sure that I will have models ready to be queried yet. We laid out our goals for Hackystat, but what are Hackystat's goals for us?
Other things that I have been wondering about: let's say, for instance, that we decide we do want to store some Twitter instances in the Sensorbase. At one point is the decision made, "Yeah, okay, this is not going to blow things up/eat Tokyo/create infinite lolcats and can send data to the sensorbase for reals"?
Eclipse is driving me crazy with its slowness, its utter refusal to clean build when I tell it to, and the otherwise batspit insane errors its been giving me. It is also making me into the whiniest coder ever, which I am sure everyone who follows my twitter is tired of. (But yes, I do accept cheese with my whine!)
Philip suggested that the slowness might come from the eclipse sensor and its communications with Hawaii. This seemed reasonable, so I took his advice and uninstalled the plugin. This helped somewhat, but I'm not sure it helps enough to justify the derth of data.
It also (sadly) did not fix the other bizarre errors I've been getting... Like, today, in which Eclipse spent an hour insisting that I couldn't implement an interface because it "wasn't an interface". It WAS an interface. Usually when one gets errors of this nature, a clean build fixes it... But several clean builds later... same error. This did not make sense to me. I added in a couple of syntax errors, which it caught, and then I removed them, which it didn't catch, even three builds later. Unfortunately, I don't remember what series of steps I took to finally make it come to its senses and recognize my interface as an interface. I'm going to reinstall the wretched, illiberal tomorrow and see if that helps. Or I may just beat my face against my monitor until it is satisfied by the ritualistic dumping of my blood and decides to behave again.
In Other News:
Additionally, this week, I have gained 10 hours of chemistry credit and killed three tomato plants by watering them with milk.