Always-On Screensharing

Design Goals

In considering the problem of how to better connect family members, I came to the conclusion that current communication methods are often abstract. When you are used to having a person living downstairs from you, having your primary point of contact become an email or text message thread may be jarring. While convenient, these communication channels are both extremely deliberate and lacking in a strong spatial relation metaphor. My goal with the design of the present system was to provide a more persistent, casual and spatial connection between estranged family members.


My design calls for a system in which users share screens in relation to one another, as if they were on a grid. Users can drag content off of their normal, private screen and onto a persistent shared screen, affording natural, off-hand communication of current tasks and interests. Rather than sending an email with a link, users can simply toss novel content up or over to their family member. They can then switch their view to the shared space in order to manipulate or view this content directly, or remain in their private space. This interaction needs to be casual and intuitive, and the system has to be both persistent and unobtrusive.


I realized early on that I had to build a prototype that actually functioned, and that this would require basic networking capabilities. The reason for this decision is that this is primarily a social system, and it needs to be tested in a social context. Users need to be able to run the system in the background during their daily routine, and use the shared screen when desired. While the usability of the window movement action was of interest to me (would it be better to have a gestural 'tossing' motion, or a drop area, or a button?), I chose not to build with these things in mind. In addition, I chose not to emphasize the issue of how to best notify users of impending messages. This latter issue would be an interesting one to address, since the notifications would have to be very subtle in order to keep the act of sending or receiving a message to feel like a major event. Several varieties of each of these could easily be incorporated into my current prototype, and studied in usability tests.

What I chose to focus on was creating a basic networked screen-sharing system between two users. In order to do this, I opted to use Jython Swing to build the interface, and a Python server to allow for communication between the clients. To mediate between these, I wrote some netcode that uses sockets to communicate with the server and client. I included some of the framework for multiple 'directions' of shared spaces, but in the end chose to hard-code two users, in order to save time. I have written a chat client in Jython using a similar setup, and I wanted to see if I could do something similar under a time constraint. In general, my experience with Swing leads me to the conclusion that is great for quickly building fairly complex behaviors, but is not the best choice for expedient assessment of look and feel. Since I wanted to test the overall system function, and communicate its purpose clearly, I decided to go with Jython + Swing over other approaches.



While my focus was on essential functionality since I wanted to be able to actually evaluate usage patterns, I did make some intuitive design decisions regarding the placement of drag and drop areas and how screen-switching occurs. To move a window to a shared space, users simply drag it over the top 10% of the screen. In a real implementation, this action would probably be done in conjunction with a holding a hotkey, since users will need to have windows near the top of the screen without unintentionally sending them to the shared space. This is something I had not considered in detail when making my initial sketches, but it became readily apparent that this would be an issue when interacting with the prototype. That said, the basic drag and drop interaction makes sense, and feels logical to me. Given time, I would implement several methods of moving content between areas (drag area, gesture drag, and contextual menu, for example), and test these. To switch screens, users click a screen-width button along the top or bottom of the screen. Note that the second user should really be clicking a strip along the bottom of their screen (due to my concept of a spatial grid), but I chose not to implement this due to the fact that this would require either multiple client varieties or additional state variables. In addition, during construction of the prototype it because apparent that the mental model of moving content *up* into the shared space was more intuitive to me than trying to move it down. This is another component I could test; I could have several direction models (perhaps divide the top of the screen into sections, or use the corners instead of thee edges) and see how this effected usage and subjective ratings. Overall, I definitely had to limit my creativity in interaction methods due to my desire to implement basic functionality. I did not get to address several of the major design questions I had, but I did get to see how the system would function as a whole.

One problem I encountered was the fact that I initially had the task of faking the entire operating system. I had to decide how best to simulate arbitrary content windows, and chose to go with basic web browser windows, something that is supported by Swing without requiring many lines of code. While I think that this aspect makes sense, the fact that things like the windows task bar, the dock, and the OSX top menus were not represented decreases the external validity of my prototype. A more developed prototype or final implementation would simulate some of these things, and would not allow itself to exist outside of the usage 'ecosystem.' Due to my desire to get something functioning fairly rapidly, I did not take these OS features into account. Being able to actually build a plugin for an operating system would greatly increase the external validity of this system, and would be desireable given more time. However, I believe my approach was high enough in fidelity to communicate and explore the basic idea well, and to reveal some obvious perks and flaws of the concept. For example, while I thought it was qutie natural moving content in and out of shared spaces, it was not clear when the other user was viewing the shared space, what they were looking at, or indeed whether a view was of the shared space or private space. Building and using a prototype like this brings these issues to come into relief. They could each be easily addressed within the Jython/Swing framework. While I could have gone with something lower-fidelity and unable to function in order to focus on granlular interactions, (movements/alerts), this approach allowed me to explore the overall act of using the system in a way that a lower fidelity prototype would not have allowed.

One area in which I found my approach lacking was in look and feel. It may have been possible to build a much more attractive prototype using different tools, and I have doubts that user testing of my prototype would not be unduly influenced by perceived attractiveness and completeness. However, I was able to gain design insight from simply using a reasonably complete instantiation of the system. Ideally, I would have been able to construct something that was less rough around the edges and also functioned, but the point of prototyping is to build only what is necessary to explore and evaluate designs within a given space. What I have is an excellent platform for doing just that.