Sunday, June 17, 2012

Mechanisms of Social Network Privacy Violations

How do privacy violations occur in social networks? In this when I say “social network,” I mean the real social network and not software like Facebook that models a social network. If you need more clarification on this definition, please refer to my previous here.

Volitional or Accidental Privacy Violations
People can volitionally or accidentally violate your privacy. An accidental violation could arise from an acquaintance mentioning a personal attribute of yours you intended to keep private to another person. This other person would have been someone from whom you chose to hide this personal attribute. Or the mention could have been intentional with the mentioning party intending you harm or just disregarding the implied privacy. Accidental privacy violations seem to occur more commonly than the volitional or intentional privacy violations even though the opposite might seem true. Volitional privacy violations generally reveal themselves more readily whereas accidental violations are not preempted because they are more inconspicuous.

Accidental Privacy Violations
Almost all accidental violations originate with the person for whom the information revealed was private. In other words, the person who is violated by an accidental revelation was the original source of the information. A person could tell another person something while assuming the recipient of the information understood implicitly the nature of the information and therefore that the information was for them only and that they were to not retransmit the information. If the recipient did not understand the implicit privateness of the information, then they might just retransmit.

Other times, accidental privacy violations originating from the owner of the information happen as a result of ignorance. The person reveals something from which others can gather or infer more information. As an example, a person could say, "I was talking to Dr. Jack yesterday afternoon and so on and so forth," and the other person might know that Dr. Jack was on a remote hike yesterday afternoon. Knowing where Dr. Jack was and that this person was talking to Dr. Jack at that time, they might surmise that this person was with Dr. Jack on the hike and this would also mean that this person and Dr. Jack were close associates. Combinatory in its construction, the privacy violation in that example illustrates a commonly executed and less readily recognized accidental or even general privacy violation.

Sometimes a single piece of information can compose a privacy violation and other times, like the previous example, the violations can come from the combination of disparate and seemingly unconnected information.

Volitional Privacy Violations
People can volitionally violate your privacy. In other words, they take a piece of information which they know you consider private and retransmit that information to people whom you would not.

We commonly refer to a subtype of the volitional violations as a rumor. One person hears something private about another person and they pass it along. Rumors, whether true or not, violate a person's privacy in a much more emotional way. It seems the intensity with which people listen to rumors motivates the violator to spread the rumor, but that lies more in the psychological realm where I should not play.

Another type of volitional violation lies on the opposite side of the seediness spectrum where the violator does not think the information requires privacy. A person might not want other people in their community to know they grew up without a father. Maybe this embarrasses the person because the separation between their parents brings shame upon them. But, the privacy violator may not think parents separating brings shame to anyone and so they casually express to another person the fact of the divorce.

The last form of volitional privacy violations arise from a true intent to harm another person or protect others. A person may have committed a crime. Another person finds out about the crime and then alerts others in the community or their circle of friends because they find the criminal potentially dangerous or they want the criminal punished. A violation of this type seems less of a violation because the criminal desired to keep their crime private. Regardless of whether or not others find the violation justified and ethical, the violation remains just that by definition. But the information in the privacy violation may have been less nefarious and simply damaged a person's reputation in a community such as a sexual orientation frowned upon by the community in general.

We have seen that privacy violations can be accidental or volitional. And, they can be autonomous in their violation or they can be combinatory. These attributes of a privacy violation describe the type of violation and the potential mechanisms in a natural social network. In software modeling a social network, like google+ or LinkedIn, these privacy violations occur with the same attributes. People have relationships, they communicate information about themselves, and other people retransmit that information. So, the potential and actual occurrences happen all the same but in a different setting than the hometown coffee shop or over the phone.

This different setting provides new mechanisms and also new aspects of the violations. People share information in an online social network the same they do person to person. But, due to the nature of online communication, transmitted information is less transient. A third party for whom information is not intended can not hear what a person says in the coffee shop at a time after it was said. If Bob tells Sally that Joe had a medical procedure, Antwerp will not hear it when he arrives at the shop a minute later. The information has been transmitted and resides only in Sally's memory. And, even her memory might lose the information making the information unretrievable.

Permanent Retrievability
Online, information transmission is more permanently retrievable. You may post something online for your current circle of friends in your social network or circle of friends. That information was suitable for their eyes and ears in that you trust them and you consider the information private outside of that circle. Later, you add a new friend to your network while forgetting that you had shared the previously private information. Your new friend could retrieve the information you would not tell them given the chance. So the information you would consider private when communicating with your new friend, your new friend now knows. Had you told your friends this private information in a physical gathering like a party, then your new friend would never have heard it because the information transmission at the party was transient. It ceased to occur as soon as your voice ceased vocalizing it.

Information shared online becomes closely permanently retrievable. Because of this, the severity, frequency, and quantities of privacy violations occurring naturally in a social network will be magnified because each revelatory mistake is close to permanent. Information revealed in different social networks with different circles of friends can be combined by others accidentally or intentionally to reveal compound attributes you would consider private. A date in your profile in one network combined with a post in another network and then joined with an email you sent to your boss, can reveal to your boss that you lied about being sick on Tuesday and in fact you were at a car show.

Software Privacy Violations
Because online social networks model natural social networks and the models created by people inherently contain mistakes, online social networking software will at some point reveal information you considered private if you posted private information on a network or networks. It seems that almost every network has violated users' privacy either by allowing hackers to enter into the network or by exposing information intended for one limited circle of friends to another circle of friends or even just one other acquaintance. These violation mechanisms are difficult for an end user to control and account for most of the distrust people have of online social networks. If you add mistakes in the software modeling the social network to a person's accidental revelations, you will find that the privacy violations do not just add up but become multipliers.

In communication within or between social networks, people have to contend with the information they shared and the audience with whom they shared the information. They have to cognizant of the nature of the information and the character of their audience. A person has to consider how to describe the nature of the information to their audience and even how they might allow that audience too retransmit the information. That is just in a natural social network. If you then add in the proxy of software modeling a natural social network, you have so many more considerations of which you may not even be aware. Look at all the various privacy settings in online social networks and their ambiguous definitions. It is no wonder accidental violations occur so frequently online. But, understanding how these violations occur will help a person recognize one or more of the mechanisms and attempt to fix the violation. The mechanisms of privacy violations in social networks are not difficult to understand and recognize but the effort involved to seek them out is not trivial.

Friday, June 15, 2012

The Nature of the Social Network versus Personal Privacy

I have read, heard, and viewed stories, debates, and complaints analyzing the degree of privacy in social networks. Countries at their upper levels have passed laws intended to corral the purveyors and users of social networks so that users of these networks can expect an unspecified and arbitrary level of privacy. I expect the legislation does not define a degree or level of privacy because degrees of privacy do not exist. Something is either private or it is exposed.

I do not think legislators or people in general consider the definition of privacy. It seems to float for most people as a concept somewhere between a person's desire for friends and their fear of enemies. But, we can define privacy. Privacy as a concept has hard and fast edges. Privacy is a state of being in which exposure of personal attributes is limited. Lets go a little deeper to some of the question begging and implications here.

Privacy is a state in which only a person can exist. A person's privacy is the sum of the attributes which they have unexposed. So, an attribute can be private, unexposed, or exposed, not private. If I want the color of my eyes to be private today, then I will wear sunglasses to hide them from view or exposure. But, the state of being private is either true or false. There is no half private or semi private condition in which an attribute can exist. The attribute is either private or not.

Attributes can start as exposed, such as my eye color. Or, attributes can start as private such as my intense affinity for small pewter dragon figurines. My eye color is something that anyone would be able to see so it is exposed. I have to hide my eye color to make it private. But, my figurine affinity is something that starts as hidden. I would have to deliberately expose my affinity for my affinity to no longer be private.

We now know what privacy is and of what it is composed. So, given a set of attributes, we might even be able to come up with a measurement and scale of privacy. But, we will have to get back to that at another time. What we now need to work out is the definition of a social network.

A social network is a collection of relationships connecting people. Social networks did not first exist on the internet. Social networks on the web only attempt to model real relationships people have. A social network as a concept is just that collection of relationships. A tribe with no electronics thousands of years ago would have a social network. The relationships between all the people in the tribe would exist even though the tribe was not logging on to Facebook. Social networks in software model and record our real relationships. Even if two people happen to meet on Facebook, their relationship does not end just because Facebook became a penny stock and was sold off piecemeal to the lowest bidders. Oh, wait, I am not supposed to let people know about my secret powers. See, I am supposed to keep that personal attribute private. The relationship can live on beyond the model in Facebook or even shift to another model like a more modern online social network.

We now know that a social network exists outside of the software modeling it but that an online or software based social network is one that models real relationships. If an online social network models real relationships, then I would expect that behavior correct in real relationships would also be correct in  the models mirroring real relationships, i.e. online social networks.

I would not tell my overly masculine and mouthy friend about my affinity for pewter dragon figurines. I would expect him to share that information in a derogatory way with the rest of our circle of friends. He can be quite mean at times. I also would not wear a shirt with my full name, social security number, and birthdate to the grocery store. I expect you also would not do such things. So, in normal life we would not expose information we wanted kept private that was originally private. What about things exposed which we want private. The tattoo of Smurfette on my forearm would normally dance in the sunlight as I was out and about. But, this exposed attribute of mine I will always make private by wearing a series of sweatbands up my arm. It works out nicely. Everyone thinks I am just really into working out.

Online I also would guard these items to maintain or create privacy. I would not post a picture of my Smurfette tattoo or discuss my affinity for pewter dragon figurines. Some at this point might protest that the nature of the web makes it such that I could privately indulge communication with others having the same attributes or affinities. They might say that in those cases I should still have the expectation of total privacy. I should be able to expose certain attributes to people without exposing those attributes to others. They would say that the web makes this possible and therefore social networking on the web differs from that in everyday offline life. This is not the case. The web only models everyday life in software accelerating positively the rate at which we can achieve certain social goals. If I want to find a group of people with Smurfette tattoos, I could quickly find other Smurfette fans online. But, I could also do the same thing offline. It might take me a long time, but I could do it. I risk the same exposure offline that I do online. I would have to search and talk to people about it to find others also wearing multiple sweatbands on their arms. Online the exposure would just have faster and broader implications because of the speed at which and the breadth of broadcast information can spread.

So, we can see what privacy is and what a social network is and how privacy plays in real life the same way it does online. But, the question remains—can we be private in an online social network?

In any social network you have to expose certain attributes in order to make connections. Could you imagine making a friend where you knew nothing about them? It is impossible. In the most basic of acquaintances you know some sort of identifier and you have some sort of communication from them. A face to face introduction delivers to each party in the meeting a whole host of physical attributes and at least first names. A connection or relationship in any social network between two people requires the exchange of attribute exposure. If a person joins a social network, then they have exposed attributes. It is not possible to join a social network without exposing some attributes. Joining a social network necessarily requires a reduction of a person's privacy in that network. The nature of a social network is such that users of the network have to give up some privacy.

Swimming requires you get wet. Socializing requires you expose personal attributes. In more succinct terms, the nature of a social network is antipodal to personal privacy.

Fear and Hope: Social Media Networks and our Personal Data

A recent survey by Avira, a software security firm, found that 84% of those surveyed worry their participation in social media will lead to their personal data being misused or stolen. The study found that the largest social network of all, Facebook, was distrusted more than any other major network. People fear how social networks will violate their privacy or misuse their personal data. People fear many types of misuse and potential negative ramifications of this misuse, misallocation, or exposure. But their ultimate fears drive their initial distrust of social media.

A person’s ultimate fear could be as simple as being embarrassed or as extreme as being a victim of a home invasion or worse. Ultimate fears could include:


Sexual or thieving predators crawl social networks. Social networks provide ample prey in a very exposed environment. Sexual predators seek out potential victims based on the attributes a user posts as well as the nature of their comments. If a user indicates some sense of emotional vulnerability, a sexual predator could use this to engage their prey. Thieves can search out timing of posts such as a recurring appointment or even a vacation mention and with some other geographic information in a person’s profile, they can figure out where a person lives and the potential robbery payoff. As reported here, a simple Facebook status update can get you burgled.


While employers are beginning to battle the conundrum of private life versus work life on a new front in social media, users are finding that they may post to their own detriment should then not censor their posts in social media. Some employers have begun to ask for job applicants’ social media credentials and doing the same for those already hired or slated for promotions. People fear their employers will find embarrassing or deleterious information in their social media presences, and rightfully so because employers are looking and seeking out information that may be harmful to the company such as exposed company information or just association with an employee who engages in activities harmful to the company’s reputation.

Education Expulsions

Educational institutions, like many companies, engage in social media surveillance of their members. They also desire to keep the image of their organization in a positive light. But members, again, understandably fear that the social media in which they engage will not protect them from this type of intrusion. Students fear they will be expelled or at least reprimanded for their online activities.

Reputation Damage

In a social media user’s circle of friends and associates, they typically have a reputation they desire to foster and maintain or potentially achieve. But, they may fear that things contradictory to that reputation could show up in social media either outside of their control like another user posting a video. They may fear that a reputation they have in one circle of friends will not be compatible with the reputation they have with another group. They fear that the overlap in their social media between these circles or groups may damage one, both, or all of their reputations.


The government is watching you. Some people fear the government intrusion into their online lives. They fear the government is profiling them with the prospect of using the profiles to direct or damage their lives at some point in the future. Many people would dismiss this notion as conspiracy minded but as reported here, the government actively, unprovoked, and without warrants profiles people on a vast scale.

Harassment and Bullying

One of the most widely reported types of data misuse via social networks is harassment or bullying. People fear they will post something and it will be used against them as a point of harassment. Or they may fear that at some point a person or group opposed to something they stand for will bully them. Like reported here, people do bully others via social media and, in the worst cases, some bullying victims have even committed suicide.

All of these fears are quite valid. The anecdotal evidence provides a substantial amount of support for the potential that people could become victims of the misuse of their information in social networks. But, how does this happen? Almost every case supporting the fears above involves information the victim provided to the social network that the perpetrator then used against the victim. The victim had no intention of providing the information to the perpetrator. The victim was either trusting or discounting the likelihood of becoming a victim. In the end information one person shared was passed to another who should not have gotten that information.

Most social media networks by the nature of their software applications, privacy policies, default settings and usage agreements fault the user. The social media networks deliver all the tools a user could need to protect themselves, right? This is certainly the case since they all provide a way to discontinue accounts. The second most effective way a user can protect themselves is by discontinuing their membership. Abstaining from social media is the most effective way a person can protect themselves.

Information passes from a user to an unintended recipient in many ways. From phishing to search engine spidering to information shared by one user is shared by another whose friends then read that information still attributed to the original sharing user. Small bits of information like geographic mentions by a user can be combined with other data from other sources like public records or a user’s friends posts to compile a more complete profile than a user actually publishes online. Every bit of information a user posts is a clue to follow to more information about that user. For example, if I was to post, “The boys of summer are coming back to my home town for spring training,” then you might easily surmise I live in Phoenix, Arizona. Then maybe someone posts on my wall that our thirty-year high school anniversary is coming up and if they could stay with me during the event. Another user could look in public records to find out what high schools were in existence thirty years ago in Phoenix and find out that there was only one. Without me really telling this other user, they now know I live in Phoenix. They know the year I graduated high school and what year I graduated. From there they have my name and they can find out what house I own in Phoenix. Now, you can assume any nefarious intentions you wish.

But, most of us want to engage with our friends via social media. We set aside our fears, engage, and temper our exposure. But, we still know deep down that we are exposing ourselves to many more people. And in turn, potentially exposing ourselves to many more people who would do us harm. Most just hope.

Sunday, June 10, 2012

The Social Network is Disconnected From the Web

There are so many web sites and pages out there—around 350 million websites and the social network does not connect to them. What percentage of those sites, pages, and elements of content have integrations and what percentage do not? The majority of web content has no social integration. This begs the question about what it means for a social network to be connected to the web. Yes, what is a social network and what is a connection to a content type we have yet to define?
The Network
A social network on the web is an application that connects people in software with specific relationships so they can interact on the web. But on the web? Is that the whole web? Or, is that just a portion of the web? Currently when it is just a portion of the web, it seems to satisfy the definition. But, what if it could be the entire web? What if it could be all sites, all pages, and all content?
The Connections
So, we know what a social network is. Now then, what does it mean to connect people? By connecting people you enable them to interact. You enable them to exchange. These connections are the binding properties that create a channel of transfer for interactions. That channel is limited by other properties of the channel. One of those properties could be time of day when interactions between the two connected parties are allowed to occur. Or, the channel could be constrained by the frequency of interaction over a period of time. Other properties can simply describe the connection. One descriptor could be used to qualify the relationship like “father” or just “friend.”
Interaction is Messaging
Interaction over these channels is just messaging. You can send a message, which could be text or media or a combination of those, to another person or persons. So when you connect with people, you then have the ability to send them messages. But, where are those messages and how are they sent, delivered, and received? From where can they be sent and where can they be received?

Messaging in a typical social network usually occurs in the domain of the social network. You use a form on that website to send a message. Associated with the form and the conversation is the original message which is the subject matter for the conversation. People send messages back and forth regarding that subject matter.

These messaging interactions seem to be pretty limited when it comes to a typical social network.
This messaging can also happen on sites or pages outside of the social network's domain via satellite integrations. These satellite integrations reside on any page or site outside the social network domain and consume web services to display content stored at the social network. This content pulled in by the integration is displayed in an area designated by the non-social network website's designers and developers. The designers and developers decide beforehand what the subject matter will be that people will message about. (This contrasts with the scenario within the social network’s website where the users choose their own subject matter when initiating a conversation. The social network users who join in the messaging in one of these satellite integrations do not get to choose the subject matter.) Usually the predetermined subject matter is the primary subject matter on a page such as an article in a news site. When users enter their comment, it is attributed to a conversation about a subject matter that may not be exactly what they intend to discuss on that page. They may want to discuss a subset of the content on that page like one of the photos accompanying the article or just a quote from the article.

Then people may also want to comment on portions of a site or page where there is no integration. What if they want to discuss the site logo or an advertisement on that page? In order to do so, a user would have to do some tricky technical maneuvering like finding the path to an image and then pasting it into their social network home page. Or they would have to use the satellite integration for which the subject matter does not apply to the logo or any content not specifically designated for that integration. The user's options do not enable them well. If the user does figure out how to comment on an image in the page by copying the path to the image and pasting it in their social network home page, they then take that subject matter out of its context in order to talk about it with their friends. Not only do they take it out of its original context, but they give it a completely different context: some page in the social network domain.
The context is the origin of the content. For example, if the content or subject matter is an image, the context is the page in which the user found the image. The context is very important to how people perceive the element on which they are commenting. A picture of a rock alone is certainly not as interesting as a picture of a rock as an inset to an article describing a new form of naturally occurring stable nuclear fuel.
Commenting on an element out of context, like where it might appear inside the social network’s domain after being shared, means so much less than commenting on that same element in its original context. When taking subject matter out of its original context, not only is it removed from the information describing it but the subject matter is given a completely different context: some page in the social network domain. So, not only is information important to subject matter missing but unrelated irrelevant information now frames the subject matter potentially increasing confusion. In a typical scenario in today’s social networks, sharing the image of the rock from the article mentioned above would mean so little to anyone who did not see the rock in its original context. The messaging would probably end up being a bunch of questions asking about the significance of the rock or worse, there would be no messaging except for the original message. People would be either confused or uninterested. Because of this “context discarding,” the conversations in the typical social network model do not have immediate access to the context of the subject matter and therefore yield much lower quality conversations than they could had the conversation occurred in the context of the subject matter.
It is possible to message in context but this in context messaging depends on the satellite integrations—the subject matter is predefined during the integration, as described previously, which can be very confusing to subsequent users who also message in the same integration point.
The web is eternal and so should the conversations on and about the web be. People in every day life often pick up conversations they had previously with another person. At one point two people will have a conversation about some piece of news. Later when they meet again, they pick up their conversation. Sometimes people have a thought that relates to a previous conversation they had with someone and they will send that thought to the other person in some manner such as a phone call, an email, or maybe just a text message. Subjects reappear in our minds all the time after having left the forefront of our consciousness. We think again about these subjects and we may have in the time that passed developed a new thought or opinion of the subject which we want to share with a person with whom we already discussed the subject matter some time in the past. This is all typical conversation scenario. But, in a social network, most of the conversations of the past are very difficult to find later. You can’t just go find that conversation again and start commenting again.
What if someone could join the conversation that you had previously without having been in the conversation at the time the conversation occurred? Longer conversations of this type occur in forums or comment sections on pages. A person can jump into a conversation that is eternal and continue the discussion with their own thoughts. It often does not matter how long the comment section has been in the page or how old the subject matter is, the conversation can continue as long as there are people who want to join the conversation.
The standard conversation format in a social network is a combination of short-term and eternal formats. The conversation can continue and more people can add to the conversation but the time between when the conversation began and the current time reduces the prominence of the conversation. A conversation that loses prominence over time due to the length of time expired only is of the short-term format. The longer a conversation has been continued, the less prominent the standard social network makes the conversation. But that does not mean that someone who could have access to this conversation may not find it to be the most prominent subject matter in their own mind at some later time. If they do want to talk about the subject matter of some conversation that occurred some time ago, it is highly unlikely they will find that conversation. That conversation will have effectively ceased to exist and he will start the conversation all over again. When they start the conversation again, they will have to comment on that content pretty quickly. Shortly, that subject matter and its associated conversation will not show up for their social network contacts. The conversation will end but not because they have finished commenting but because of some arbitrary time constraint.
All of this affects our ability to make like-minded connections—connections with people who think similarly about interests the parties have in common. If when browsing we see content that interests us and about which we have a particular opinion, how do we find someone else who also is interested in this content and may have an opinion we appreciate? In the typical scenario we would not be able to find such people easily. In the social network silo’ed pages, you might bump into someone but only if you happened to match up with the timing of someone sharing that subject matter you find interesting and that you were close enough in their connections to see their comments and learn their opinions. The design of the messaging in most social networks makes it more difficult to find like-minded people with which to extend our circle of friends. When we have actual connections with like-minded people, our conversations can be deeper, lengthier, more intense, and more meaningful than the conversations we have with people in passing. When people comment in a comment section of a website, they rarely are commenting with people they know. And, when they have conversations with the people to whom they have explicit connections in their social network, the conversations rarely match with the subject matter that interests us most. Our offline friendships are often driven by situational contexts rather than subjects of interest. For example, most people are connected in their social networks with their family. Other situational contexts that lead to online social network connections are church membership, common employers, or random public meetings like at a bar on Friday night. Does this mean that the conversations they will have with these contacts will be about subjects in which both users are equally deeply interested? It is not likely. The conversations will most likely be about subjects in which the participators are equally less interested or disproportionately interested.
Our connections typically do not lead to deep meaningful conversations and the social network is disconnected in such a way from the rest of the web that it makes it difficult to make connections based on common ground between people.

The disconnect
As a user traversing the web, you never know if someone has shared or commented on any part of the web unless there is some integration, and integrations are rare relative to the number of websites, web pages, and elements of content. For most of the web and all of its subject matter and elements of content, users do not know what of that content may have been the subject of some conversation in a social network. Looking at a particular photo, it may have been shared 10 times on some social network but you would never know. Not only do you not know how many times it has been shared but you do not know by whom or what the conversations may have been. Conversations that social networks have for any particular subject matter are disconnected from the subject matter in it’s own context: the web.
If you were to pretend that a particular image, that had been shared and discussed many times on the social network, were a door and you could open that door to find all of those conversations, then you would have a connection again. But those conversations would unfortunately still be subject to the lack of context with which they were previously scarred.
This door does not exist. The connections from the content on the web to the conversations about that content rarely exists and when they do they are poor connections. The social network is disconnected from the web.

Wednesday, May 16, 2012

A new social network and app

I have created a new social network and an app to deliver that network across the entire web.

I built because I could not stand the social networks out there. The are so bad that I had to write a couple papers about the networks in general. I will post those soon. I found I could not do the social things on the web the way I wanted to do them. So, I made to do exactly those things. But, the things I wanted to do were so different than the way the other networks operated that I had to create a totally new network rather than just using the networks that are out there. I will likely in the future integrate with those other networks. It seems it will be beneficial to do so but for now the network is its own network. It is not a meta-network like Pinterest. It is a bit similar to Pinterest but the concept was derived and designed from a much deeper understanding of how people interact socially in general and how social interaction should happen on the web. In fact, I did not even know about Pinterest until someone said to me after looking at, "I like your implementation much better than Pinterest's."

Then someone I know was, unbeknownst to me, using Pinterest. They started complaining to me about the things Pinterest would not let them do. For every one of their complaints, had the solution. It was quite funny and certainly made me smile.

So, please take a look at and enjoy. Hit me back with comments and suggestions. I love it so far but there is much more I am going to be adding to .

Happy tagging!

Saturday, November 12, 2011

My Favorite Javascript Method for Removing Array Duplicates

var dedup = function(a)
    var b = [], //create a replacement array
        i=a.sort().length; //get the dups together
    while(i--) //loop down the original array
        // they are bunched, so copy only non matches
            b[b.length] = a[i];
    return b; // return the replacement array
// test it
console.log('Original Array: bob,sally,9,sally,george,george,andrew,23,230,\'23\'\n' + 'Deduped Array:'+ dedup(['bob','sally',9,'sally','george','george','andrew',23,230,'23']));

Tuesday, November 16, 2010

Mapping Classes to Flexible Schemas

The Problem: Schemas change signature over the lifetime of the schema. So, at a single record level, the schema for the entire dataset can be modified just by adding a property to the single record. Now you have a schema that has some clean rules implicitly with a single record that is an exception to the clean rule. The rules just bent a little bit and if you had a class that mapped getters and setters or a DAO to that schema, it is now broken for the exception record.

The solution: Allow the classes mapped to the schema to mutate according to the mutations in the schema. So, if  you could have a class that when instantiated looked at its model and was able to inspect or discover according to that model what the class' data access signature should be and allow that class to create that signature, then you would have a class that solves the problem of the mutating signature.

Case in point: A form where n custom elements are allowed. So, if I have a form where people are allowed to add favorites and those favorites can be of any user specified type, then I have a model being created according to an unknown or less known context. So, the user says "I have a favorite Movie, and I have a favorite Song, and I have a favorite Californian Bordeaux, then how in the world could we predict that and create a model according to that without a bunch of silly aggregation at the class level? Instead, we let the user modify their User model which at first looked like:
{"FirstName":"bob","LastName":"smith", "favorites":[]}
and now that they modified it, it looks like:
{"FirstName":"bob","LastName":"smith", "favorites":{"Movie":"Troll","Song":"Love Song","Northern California Bordeaux":"Frogs Leap"}}

What we do is we have our class of Person inspect this record and decide what its getters and setters will look like like so:
getFirstName, getLastName, getFavorites, getFavoritesMovie.

I ran into this issue while using Node.js and MongoDB. I was using the Server Side Javascript Library from MooTools. In a subsequent post I will show how I solved this issue within this technical context but I will show how simply it can be simplified using just MooTools and a mock document from MongoDB.