Continuations: The (Re)Convergence of User Experience?
David Semeria
· 2 months ago
Or, perhaps more fancifully, a new open "meta-net" will emerge.
Right now, information already *is* behind a wall (FB, Amazon, IMDB, Wikipedia) etc. The wall is the domain where the data is stored. It's still a wall, even if the door is (mainly) always open.
Twitter is half-way towards the "meta-net" because you can access the information through an API, and by-pass the site itself. But it's not truly open because twitter (rather than the users) ultimately control the access.
The next step would be for users to be able to 'host' their data wherever they choose, and allow other users and applications selective access.
albert
· 2 months ago
I guess that's in part what you are trying to accomplish with lmframework and also Terry Jones with fluiddb.
terrycojones
· 2 months ago
Hi Albert & David (thanks for pointing me to this - I don't read blogs anymore :-))
I like David's comment. From my POV, applications should not have the last word on control of (our) information. Even when an application provides an API, you're still living in a walled garden to a large degree. You can only do (via the API) what's been anticipated, what's allowed, what they've bothered to implement, at the speed that they'll let you, etc. And if you piss off the API provider they can simply shut the door, or do more subtle things like rate limit you. For all these reasons I think application APIs are an intermediate step on the road to real openness and perhaps to the kind of convergence that Albert describes. In my mind, that kind of convergence (done properly) means *anyone can play*, and they don't have to ask permission or have their needs anticipated in order to do so. Plus, arriving late is not detrimental - no-one gets to set the rules, tilt the (data) playing field, etc.
FluidDB is indeed going after all this. And we're doing it by, in a sense, giving the data its own API. By changing how the underlying objects are owned, by leaving the last word on what happens to data in the hands of regular people (you need to be a programmer to start with, but that will change).
I'd love to see this thread continue, so feel free to followup!
albert
· 2 months ago
This is a great vision for FluidDB. The key will be to have folks build applications that don't want to "hog" the data.
terrycojones
· 2 months ago
If you wanted to hog data, you probably wouldn't use FluidDB in the first place :-)
I think there will be many FluidDB apps that will maintain objects with some things visible, some things not, some readable by certain others, etc. The permissions system lets you do that quite simply, as well as more exotic things like tags that can be used for audit purposes (a la WORM), including being able to verify that they can no longer be altered, etc. You also get flexibility after deployment: to add tags that help others find your data, to make things more or less restrictive, to adapt to your own app's needs, to encourage other apps to do their thing with their tags, etc.
It might also be worth explicitly pointing out that I'm looking at the Convergence issue from the POV of data, not UI. I don't think we need a "re-" prefix in the case of data :-)
OK, too many words, I know. Back to hacking. I'm doing something cool with Twitter data - partly to give people a fun/useful utility, but more to show programmers some FluidDB data and give them some ideas about how they can join in the fun.
David Semeria
· 2 months ago
As you know Terry, I also share your overall vision.
Where we differ is on the role of structure. You don't like it, I do. I believe that a platform-wide record description format makes data more inter-operable, not less.
LM only has one general-purpose data structure (for all applications), and the record descriptions are stored and handled separately. Cooperating applications can identify the equivalent of the "lowest common denominator" in their respective record descriptors, so as to facilitate the optimum level of data exchange (which is the underlying basis for inter-app integration).
LM is DB agnostic; its role stops just beyond the server. This explains why I was so excited when I first heard about Fluid DB, since it could represent an ideal back-end for LM applications - especially considering all the sharing possibilities flowing from the single-instance model.
Cross-app data sharing represents a tremendously challenging and promising space. I look forward to combining Fluid DB's ground-breaking ideas on data management with LM's vision of web app compatibility.
terrycojones
· 2 months ago
Hi David
I love structure. Where would we be without it? I just think it should emerge, not be pre-ordained. And being serious about a statement like that leads, at least indirectly, to the main architectural innovation of FluidDB - that objects have no owner. Or that's where it led me. I think you look at FluidDB and complain a little because you see a primordial soup. I look at it from a longer-term position, knowing that if it's successful it will *necessarily* be structure rich.
> LM only has one general-purpose data structure (for all applications),
The same is true of FluidDB.
> and the record descriptions are stored and handled separately.
Separation of data is something I think you have to be very careful about, and to avoid if possible. Who gets to decide what's in a "record description"? When? Etc. Taking that step - creating a distinction between types of data, in which one is more important, more "meta", more privileged etc., than the other - is taking a real step down a slippery slope. I don't think that's at all obvious, which helps to explain why so many programmers have been tempted to set up systems with a distinction between data that seemed important and logical to *them*, but which ultimately meant the system was of limited flexibility. I wrote about that at some length (as usual) on my blog.
I'd add that I mean this at a low level. If apps, or an app framework like LM (as far as I understand it) were built on an architecture like FluidDB's, then the data is not held hostage by the app or the framework. There's still an API into the data, and people and other apps can have the freedoms that FluidDB is trying to provide. That's all well & good, and is an example of me liking structure :-)
I hope that makes sense.
David Semeria
· 2 months ago
It certainly does. Perhaps I was wrong to infer that since Fluid DB is structure agnostic, you are too. My apologies.
But I was actually trying to point out how Fluid DB's extreme flexibility can lead to it being employed in many different contexts.
For a variety of reasons, LM must pin its colours to one structure mast, or another. A library routine which has to decide whether it's okay to drop an element from one data source into a folder owned by another must have some starting point for deciding the extent the two sources are compatible, if at all.
The fact that Fluid DB is flexible enough to allow different higher-order structures (representations) to co-exist in the same space is a remarkable feat, and suggests a wide variety of future use cases.
As to design-time decisions creating future constraints, I agree entirely. LM allows for record definitions to be extended using a method similar to class inheritance. Two record definitions can share a common ancestor, which would form the basis for the data interchange. It's a big step up from SQL tables, though it's not as elegant as Fluid DB's approach. But, as I've just said, a framework's libraries do require at least some degree of specificity.
Developers are free to reuse and extend (or contract) each others' definitions, but they must bear in mind that any data transfers will be based on the lowest common denominator (ancestor) principle. This is the compromise we have made to allow as much flexibility and extendability within the confines of interoperability.
And from nuts to primordial soup :-) That reference comes from a long discussion on the Fluid DB list when I was trying to suggest ways of guaranteeing that the structure you predict will emerge, is allowed to do so as efficiently as possible. Again, it was not meant as a criticism - it was just a colourful way of defining the starting state.
Since I'm not sure how keen Albert is on hosting this type of philosophical debate (which has now wandered quite far from his original post) I think I'll leave it there.
As always Terry, it's refreshing and fun exchanging views with you.
terrycojones
· 2 months ago
Who is Albert?
albert
· 2 months ago
I love hosting this kind of debate!
terrycojones
· 2 months ago
I think the host should offer us all a drink! It's midnight here.
Who needs smileys?
albert
· 2 months ago
Yes - drinks are on me!
David Semeria
· 2 months ago
That's certainly one of the long-term goals for LM, but it will be quite a while before we get there.
I don't want to speak for Terry, but since there is only one instance of Fluid DB, I suspect that it is more likely to contain metadata (tags) than huge underlying datasets.
LukeG
· 2 months ago
It's interesting. They're all moving toward some sort of search agent (right?), where you get what you want, when you want, where you are. I've always thought the importance of the browser in the continued evolution of the search market was popularly undervalued - it's essentially the lowest layer in the consumer's internet stack that has low enough switching costs to be a(n almost) competitive market (e.g. vs bundled hardware + OS). If that's actually the case, browser innovation should be fairly rapid & profound, and not just another distribution channel for the big search players. So...the reconvergence of the web user experience should pull power away from the browsers.
Two aspects of the (formerly?) growing distribution of web user activity are the wild proliferation of sites but also growing consumer comfort with managing a network of destinations and resources, which stands in pretty start contrast to the eras of the walled gardens (e.g. AOL) and then the major portals (e.g yahoo). This fragmentation seems (seemed?) to create significant opportunity across a bunch of markets - it's a large part of what http://jobsyndicate.com was meant to explore in our business, for example, and one of Facebook's larger strategic pushes - FB Connect - was made in order to ride this, too, right?
I hadn't explicitly recognized this re-convergence until you pointed it out. Definitely something to think about. Interesting, too, to think about how mobile + social are integrated here.
Thanks for the post.
albert
· 2 months ago
Great comment. Meant to address the role of the web browser and forgot. Kikin is headed for a "player" (user agent) inside the browser on the theory that there is a lot of other stuff to building a browser. Will be interesting to see where http://rockmelt.com winds up.
kidmercury
· 2 months ago
yes. great post albert. you touch upon what i view as THE disruptive opportunity on the web; i don't see any of the big players (save perhaps facebook but do you really consider them a big player, in the same league as the others noted?) being well positioned for this. the big players are all invested in infrastructure assets; i think the forthcoming disruptors will be investing in partnerships and deep API integration, and perhaps choosing to leverage commoditized infrastructure technology external to the firm instead of making it an internal priority, like the incumbents do. kikin, for instance, fits the disruptive paradigm i am referring to, in the sense that they do not appear to be investing as much as many others would in infrastructure, doing things like focusing on leveraging the user's computer and having things run locally.
but the reconvergence you speak of is also going to force people to address what has been avoided for so long: the dreaded copyright mess. who owns what only gets sloppier in this world. and this sloppiness will lead to fights, especially when folks start to try to monetize to full capacity.
albert
· 2 months ago
Definitely consider facebook to be in the same league. Agree that this will raise further "ownership" issues.
LukeG
· 2 months ago
Your timing's pretty excellent, too - Mozilla introduced Raindrop (http://labs.mozilla.com/raindrop) right about the time you posted this.
albert
· 2 months ago
I saw that -- same thrust for messaging. I just love how unsettled everything still is!
All Comments
Right now, information already *is* behind a wall (FB, Amazon, IMDB, Wikipedia) etc. The wall is the domain where the data is stored. It's still a wall, even if the door is (mainly) always open.
Twitter is half-way towards the "meta-net" because you can access the information through an API, and by-pass the site itself. But it's not truly open because twitter (rather than the users) ultimately control the access.
The next step would be for users to be able to 'host' their data wherever they choose, and allow other users and applications selective access.
I like David's comment. From my POV, applications should not have the last word on control of (our) information. Even when an application provides an API, you're still living in a walled garden to a large degree. You can only do (via the API) what's been anticipated, what's allowed, what they've bothered to implement, at the speed that they'll let you, etc. And if you piss off the API provider they can simply shut the door, or do more subtle things like rate limit you. For all these reasons I think application APIs are an intermediate step on the road to real openness and perhaps to the kind of convergence that Albert describes. In my mind, that kind of convergence (done properly) means *anyone can play*, and they don't have to ask permission or have their needs anticipated in order to do so. Plus, arriving late is not detrimental - no-one gets to set the rules, tilt the (data) playing field, etc.
FluidDB is indeed going after all this. And we're doing it by, in a sense, giving the data its own API. By changing how the underlying objects are owned, by leaving the last word on what happens to data in the hands of regular people (you need to be a programmer to start with, but that will change).
I'd love to see this thread continue, so feel free to followup!
I think there will be many FluidDB apps that will maintain objects with some things visible, some things not, some readable by certain others, etc. The permissions system lets you do that quite simply, as well as more exotic things like tags that can be used for audit purposes (a la WORM), including being able to verify that they can no longer be altered, etc. You also get flexibility after deployment: to add tags that help others find your data, to make things more or less restrictive, to adapt to your own app's needs, to encourage other apps to do their thing with their tags, etc.
It might also be worth explicitly pointing out that I'm looking at the Convergence issue from the POV of data, not UI. I don't think we need a "re-" prefix in the case of data :-)
OK, too many words, I know. Back to hacking. I'm doing something cool with Twitter data - partly to give people a fun/useful utility, but more to show programmers some FluidDB data and give them some ideas about how they can join in the fun.
Where we differ is on the role of structure. You don't like it, I do. I believe that a platform-wide record description format makes data more inter-operable, not less.
LM only has one general-purpose data structure (for all applications), and the record descriptions are stored and handled separately. Cooperating applications can identify the equivalent of the "lowest common denominator" in their respective record descriptors, so as to facilitate the optimum level of data exchange (which is the underlying basis for inter-app integration).
LM is DB agnostic; its role stops just beyond the server. This explains why I was so excited when I first heard about Fluid DB, since it could represent an ideal back-end for LM applications - especially considering all the sharing possibilities flowing from the single-instance model.
Cross-app data sharing represents a tremendously challenging and promising space. I look forward to combining Fluid DB's ground-breaking ideas on data management with LM's vision of web app compatibility.
I love structure. Where would we be without it? I just think it should emerge, not be pre-ordained. And being serious about a statement like that leads, at least indirectly, to the main architectural innovation of FluidDB - that objects have no owner. Or that's where it led me. I think you look at FluidDB and complain a little because you see a primordial soup. I look at it from a longer-term position, knowing that if it's successful it will *necessarily* be structure rich.
> LM only has one general-purpose data structure (for all applications),
The same is true of FluidDB.
> and the record descriptions are stored and handled separately.
Separation of data is something I think you have to be very careful about, and to avoid if possible. Who gets to decide what's in a "record description"? When? Etc. Taking that step - creating a distinction between types of data, in which one is more important, more "meta", more privileged etc., than the other - is taking a real step down a slippery slope. I don't think that's at all obvious, which helps to explain why so many programmers have been tempted to set up systems with a distinction between data that seemed important and logical to *them*, but which ultimately meant the system was of limited flexibility. I wrote about that at some length (as usual) on my blog.
I'd add that I mean this at a low level. If apps, or an app framework like LM (as far as I understand it) were built on an architecture like FluidDB's, then the data is not held hostage by the app or the framework. There's still an API into the data, and people and other apps can have the freedoms that FluidDB is trying to provide. That's all well & good, and is an example of me liking structure :-)
I hope that makes sense.
But I was actually trying to point out how Fluid DB's extreme flexibility can lead to it being employed in many different contexts.
For a variety of reasons, LM must pin its colours to one structure mast, or another. A library routine which has to decide whether it's okay to drop an element from one data source into a folder owned by another must have some starting point for deciding the extent the two sources are compatible, if at all.
The fact that Fluid DB is flexible enough to allow different higher-order structures (representations) to co-exist in the same space is a remarkable feat, and suggests a wide variety of future use cases.
As to design-time decisions creating future constraints, I agree entirely. LM allows for record definitions to be extended using a method similar to class inheritance. Two record definitions can share a common ancestor, which would form the basis for the data interchange. It's a big step up from SQL tables, though it's not as elegant as Fluid DB's approach. But, as I've just said, a framework's libraries do require at least some degree of specificity.
Developers are free to reuse and extend (or contract) each others' definitions, but they must bear in mind that any data transfers will be based on the lowest common denominator (ancestor) principle. This is the compromise we have made to allow as much flexibility and extendability within the confines of interoperability.
And from nuts to primordial soup :-) That reference comes from a long discussion on the Fluid DB list when I was trying to suggest ways of guaranteeing that the structure you predict will emerge, is allowed to do so as efficiently as possible. Again, it was not meant as a criticism - it was just a colourful way of defining the starting state.
Since I'm not sure how keen Albert is on hosting this type of philosophical debate (which has now wandered quite far from his original post) I think I'll leave it there.
As always Terry, it's refreshing and fun exchanging views with you.
Who needs smileys?
I don't want to speak for Terry, but since there is only one instance of Fluid DB, I suspect that it is more likely to contain metadata (tags) than huge underlying datasets.
Two aspects of the (formerly?) growing distribution of web user activity are the wild proliferation of sites but also growing consumer comfort with managing a network of destinations and resources, which stands in pretty start contrast to the eras of the walled gardens (e.g. AOL) and then the major portals (e.g yahoo). This fragmentation seems (seemed?) to create significant opportunity across a bunch of markets - it's a large part of what http://jobsyndicate.com was meant to explore in our business, for example, and one of Facebook's larger strategic pushes - FB Connect - was made in order to ride this, too, right?
I hadn't explicitly recognized this re-convergence until you pointed it out. Definitely something to think about. Interesting, too, to think about how mobile + social are integrated here.
Thanks for the post.
but the reconvergence you speak of is also going to force people to address what has been avoided for so long: the dreaded copyright mess. who owns what only gets sloppier in this world. and this sloppiness will lead to fights, especially when folks start to try to monetize to full capacity.