Next Named Storm – No SQL

Posted on February 18, 2013 by Gordon Allott

This post is actually a response to a very thoughtful article by Merv Adiran and Ted Friedman over at Gartner. Their point, which is a very important one, is that a platform does not make a solution. For example, some keen developers are using Hadoop as an integration tool. I mean it’s awesome! I remember using MS SQL as an integration tool. Cool? Yes. But a solution? No. It’s a bag of snakes. Pretty much poison for a well-functioning enterprise.

A bit naughty as it may be this is really important stuff. Why? Disruption is happening here. Technology-wise we have seen this again and again. And, the No SQL technology, although somewhat slow moving, is a whopper which has a very high probability of changing the status quo.

To me it looks like this:

Data Integration: System to System
MDM: Master Data Management
ETL: Extract, Transform, Load
B.I.: Business Intelligence “a-la Cube”

The United States of Data Applications is our neighborhood. Most vendors in the space play seriously in some and dabble in others. There’s always a lot of confusion about what K3 does versus Informatica versus Mule, Versus Biz Talk, etc etc. Widely differing approaches here…that’s another blog post.

The challenge is this disruption is coming straight to the United States of Data Applications. It is still uncertain where the No SQL hurricane will hit. Could curve left…could curve right. But when No SQL starts taking shape from a platform to solutions…it’s going to be a different world.

The Year of the Snake…and Trade Break

Posted on January 15, 2013 by Gordon Allott

I’m not big on New Year resolutions or on predictions. But in the case of Dodd-Frank and EMIR in Europe it’s hard to resist. The die has been cast and SDRs are a real thing now.

Here Come the Trade Breaks

Setting aside for the moment the master strokes about what DF and EMIR are supposed to be about, let’s talk for a moment about the hands on, down in the trenches effect of what these rules are actually causing.

A trade clean up of epic proportion.

It sounds like a great and practical idea. Perhaps an idea that many firms agree have been put off too long. But there is a problem. A big one actually…this is the commodities business.

In the commodities business we have legacy systems. No I don’t mean just in-house built systems. I mean vendor systems that were architected in the 80′s and 90s. Lots of them out there, sold as new, patina upon patina, with the same fundamental architecture they started with. I was working with a vendor system at a client. One of our younger developers was pulling his hair out asking why, why did they do this?! The answer unfortunately is the same as the reasoning for popped collars and hair gel in the 80s. “It was the thing at the time.”

Immediately on everyone’s plate is how to get trades out of these old architectures and get them properly into the SDRs. This turns out to be, at best, like data gymnastics. At worst mad contortion. If the data is a spaghetti-works to begin with the probability of a trade break goes through the roof. Surveying the dozen or so systems we have connected to the SDRs better than 3/4 of those have a significant amount of transactions that have been “shoehorned” into the system. This is a recipe for a trade break downstream with ICE/DTCC/CME or whatever SDR has been chosen. These systems are somewhat unforgiving, and as a result unless they come across perfect they bounce.

Forget the Year of the Snake. This is the year of the Trade Break.

Architecture | Chevron Has More Data Than Google

Posted on June 18, 2012 by Gordon Allott

Ponder this: “Chevron Has More Data Than Google.”

I know …I know…..That was a rumor I heard that the CEO of Chevron said at CERA Week.

It’s really one of those comments that made me scratch my head. I don’t know if Watson actually said this. And after my knee jerk “no way” reaction, I started thinking something else. “What if this is true?”

Further ponder this: “Data is the New Oil.”

I’ve heard this idea flung about quite a bit lately. And considering that I have been both in the energy business and technology for 20 years I am about to abuse this analogy like it has never been abused before.

Here’s a crash course in oil production. In the oil business, oil development is broken down into three states of production.

P3 is “Unproven or Possible” (read the oil is probably there),
P2 is “Proven Undeveloped” (read we know the oil is there but not producing yet) and;
P1 is “Proven Developed” (read the oil is flowing).

Remember we are talking about real oil under the ground. Chevron has spent decades and decades figuring out how to transform the “maybe” oil into “produced” oil or P3 to P1. So let’s use this concept to the new oil.

If “Data is the New Oil,” Chevron is potentially sitting on top of the Saudi Arabia of data. But where is the monetary value in data??? Value is created by moving from data “possibilities” to “data monetization.” Some rigs cost $400,000 per hour to run. If you can shave a couple hours off of any one of the thousands of rig processes that a driller uses in any of the thousands of wells drilled we are far into the billions saved. Anyone in the oil business knows that there are thousands upon thousands of processes. We can do them better with the right data and analytics.

But, is that it??? Just cost savings? My instinct is no. So much of our day to day lives is plain old grunt work. If you look at any software process, 80% of the time is just getting data TO the process. It’s the same for work processes. So much time is spent getting ready to do the real work. I can’t help but think that harnessing this data may really change how people work on a day to day process.

But how do we get to there from here? Two immediate hurdles. Challenge one is that most of that data is locked up in older architectures. We are going to need to really re-think integration. The second challenge is processing. See Big Data architectures like Hadoop. This is why Chevron is playing around with it. http://blogs.wsj.com/cio/2012/06/05/chevron-explores-open-source-using-hadoop/

Opinion | Call for Visionary CIOs, CTOs and COOs

Posted on June 14, 2012 by Vivek Pathak

I’ve worked inside of a number of large trading shops now, both as an employee and as a consultant. As a result I’ve worked with some really smart people…people who can tell you all the minutae of how to price complex options, people who can build technology systems to compute the value of thousands of trades at the push of a button and leaders who run globally distributed groups serving a complex matrix of stakeholders all with different needs. Quantitatively, it’s been a daily learning experience. Qualitatively, it’s like watching the head of snake trying to swallow a mouse too big and giving itself whiplash while trying to eat.

The process pretty works like this…

A trader or sales person (commonly known as the Front Office) needs something from his technology systems to make his job easier. Could be as simple as the way he views his positions to something more difficult like explaining the factors leading to the change in his portfolio value overnight. Almost inevitably, the Front Office person will snap their fingers and work with someone from IT to make the required change(s). The IT developer, buried under a growing list of such requests, greases “it” to come up with the fastest solution. It’s like watching someone row a boat with holes in the bottom…they are rowing at full tilt just to stay afloat.

The IT Manager is measuring the Developer based on how much they have ”done” for the Front Office (not always how they have done it) and the pressure only adds incentive for short cuts and band-aid solutions. What’s the net effect for the enterprise?

Front Office person is temporarily satisfied. Developer goes back to addressing their laundry list of such fixes and the IT Manager feels like they are running a successful team. Except that building systems with limited foresight is like putting up scaffolding held together with thread. Once it starts to rattle, you need to prop it up. And the fastest way to prop up enterprise systems is to throw people at the problem.

So the next time that Front Office person requests an incremental change to the original fix, the effort is double as they must revise the original fix to cater to the new request and then actually complete the new request. Extrapolate this out a couple years with many Front Office folks making many such requests and you get:

Underperforming tools for Front Office users
Computer Engineers whose jobs are reduced to holding up scaffolding
Middle Office folks whose requests never get addressed

As the scaffolding becomes more fragile the relationships between these groups can become fractured. Turns out that it doesn’t have to be this way.

Global IT spent this year is going to be around $1.2 trillion (Gartner) and 25% of it ($300billion) will be spent on building system interfaces. So we built a product, K3, that aims to remove the scaffolding around system interfaces. Open source components for developers to build interfaces and a simple GUI for Middle Office people to manage the data flow thereafter. Empowering users to self-drive allows the Middle Office to handle 80% of the Front Office requests around interfaces and Developers to focus on being engineers instead of data janitors.

Results for the Enterprise? Front Office folks can focus on revenue, Developers can return to being Engineers on strategic initiatives and Middle Office people can raise their hand with the hope of being noticed. Now everybody hug.

If you’re a visionary CIO, CTO or COO who is looking to build an enterprise and get past the scaffolding, email me. We need to talk.

Opinion | Greasing ‘it’ – Divide between Business and IT

Posted on June 10, 2012 by Gordon Allott

I was having coffee with a friend of mine who is a commercial pilot. With as much flying as I do it’s hard not to ask him a ton of stupid flying questions. We got onto the subject of landing. His point was, “Too many pilots try and grease the landing.” A lot of us cattle customers judge a pilot’s ability by how smoothly he or she is able to put the plane on the ground. We just don’t have anything else to go on. But the reality is that in order to “grease” the landing you’ve got to give up precious runway. “The landing zone is there for a reason…so you have lots of runway left in case something goes bad.”

Greasing It

This got me thinking about where and how we, in the IT business, get caught in trying to grease it.

Sometimes when we go to clients there is a big divide between the business and IT. The preverbal locked cockpit door behind which all the IT magic happens. The business is kept in the dark about exactly what it takes to get projects done the right way. This facilitates an attitude of just getting things done quickly and quietly for the business. But it is also really fertile ground for taking short cuts that have a longer term impact on the business.

There is a reason 92% of integration is custom code. Those messy point to point integrations that are a pain in the neck to manage. The IT managers greased it.

Opinion | Enterprise Guys in a Social World

Posted on May 22, 2012 by Gordon Allott

So we are at TechCrunch Disrupt in NYC this week. K3 demo’d as a finalist in the Disrupt Battlefield Competition yesterday to an ocean of MacBooks. As far as we can tell we are only one of two enterprise applications here. Everything else? SOCIAL. Social Consumer, Social Media…social…social … social… (Marsha, Marsha, Marsha!)

Even among the social apps there is something of a dark specter lurking amongst the

Where's the Money?

Startup Alley tables. How much more can the “C” category actually consume??? And moreover, how can you MONETZE it???

We enterprise apps guys live and die by “value prop” and monetization strategy. K3 reduces maintenance costs of interfaces by 50% and virtually eliminates lost data problems associated with custom code interfaces. It’s a $300 Billion dollar market. That’s how we built it. That’s how we sell it. That’s how our clients come to love it.

So, let’s think about the social value prop. For most of these apps it is about harnessing a social community of trust. But, if you can’t monetize the community of trust it boils down to simple entertainment/advertising model. Pinterest…love it…its entertainment. Facebook…love it, but its entertainment. Unfortunately, the rabbit like propagation of social entertainment, based entirely on advertising revenue, has bit the hand that feeds it by turning advertising into total white noise.

Let’s look at Facebook. I’ve created a highly trusted community there. My buddy Daniel is a prolific reader. When he finishes a book on Kindle, I don’t even look at the title. I just buy it. But what’s FB getting out of this? Nada. But that’s the trusted interaction social apps HAVE to monetize.

If there is anything I believe in, it’s the creative power of the people at this conference. The next Google, the next Facebook is going to have to come up with a brand new revenue model that taps the trusted community.

BroadPeak Think

Innovations in Interface Management, Trading Technology, Architecture and other tidbits

Category Archives: Opinion

Next Named Storm – No SQL

The Year of the Snake…and Trade Break

Architecture | Chevron Has More Data Than Google

Opinion | Call for Visionary CIOs, CTOs and COOs

Opinion | Greasing ‘it’ – Divide between Business and IT