Category Archives: Ultra Messaging
Simplistic Approaches to Data Federation Solve (Only) Part of the Puzzle – We Need Data Virtualization (Part 2 of 3)
In my last post, I introduced the concept of data federation, which for now I would like to differentiate from data virtualization – a term that I’ll bring into focus in a bit. But first, we explored two issues: data accessibility and data latency. Within recent times, the sophistication of data accessibility services has matured greatly, to the point where one can somewhat abstract those accessibility services from the downstream consumer (or “reuser”) of data. (more…)
Remote Data Collection and Transformation – with Ultra Messaging Cache Option and B2B Data Transformation
Sometimes when I drive past an electronic tollway collection sensor, I wonder about the amount of data it must generate. I’m no expert on such technology, but at a minimum, the RFID sensor has to read the chip in your car, and log the date and time plus your RFID info, and then a camera takes a picture to catch any potential violators. Now multiply that data times the hundreds of thousands of cars that drive such roads every day, times the number of sensors they pass, and I’m quite sure this number exceeds several million messages per day. (more…)
In a recent post: Remove the Restrictor Plate with High Performance Load Balancing, my colleague Jeff Brokaw compared the high performance architecture of Informatica Ultra Messaging to the removal of the carburetor restrictor plate on a NASCAR racing engine to increase airflow and speed. Ultra Messaging has removed the “restrictor plate” in that it provides direct peer-to-peer communication between applications with no intermediary brokers – thereby delivering extremely high and sustained throughput rates at low latencies. (more…)
Similar to the way that a carburetor restrictor plate prevents NASCAR race cars from going as fast as possible by restricting maximum airflow, inefficient messaging middleware prevents IT organizations from processing vital business data as fast as possible.
In a recent post: Informatica Ultra Messaging Software Supports Capital Markets Reforms, I discussed the technology implications of the OTC derivatives (swaps) market moving to electronic trading as mandated by the Dodd-Frank Act (DFA) in the US and the European Market Infrastructure Regulation (EMIR) in Europe. One area where new technology infrastructure will be especially critical is in the creation and operation of “exchanges” for electronic swaps trading, similar to what is used for equities and other asset classes. In the language of the DFA, such exchange venues are called Swap Execution Facilities (SEFs) and are defined as “a facility, trading system or platform in which multiple participants have the ability to execute or trade swaps by accepting bids and offers made by other participants that are open to multiple participants in the facility or system, through any means of interstate commerce.” This of course includes capturing orders electronically, matching bids and offers, executing the trades, and providing connections to central clearing houses. And perhaps nowhere else in the new ecosystem is the expected growth in message volumes and associated need for new messaging middleware technology more evident than here. (more…)
In the past, the term latency has been largely ignored in the IT world, with the exception of network engineers and algorithmic trading experts. But today, there is compelling evidence that latency is an important metric for every business that runs a website, or that deploys Rich Internet Applications (RIAs), because even small delays in presenting data show a clear pattern of pushing customers and readers away.
Interesting data, replicated by multiple sources (including Bing, Google, and Amazon) show that slow-loading pages can cause the viewer to lose focus and potentially even click on something else, possibly never to return.
For instance, on search results, a delay of just .5 second chases away up to 20% of the traffic and revenue. As it says at this O’Reilly Radar post, “delays under half a second impact business metrics”.
Our first post in this series on Efficiency covered the high-level performance benefits of super-efficient messaging software, whether you measure for latency or throughput, since efficiency is the property of software that provides performance. “Ultra-low latency” is just another term for extremely fast, lean, efficient execution. For more, see the post: Ultra Messaging is Also High-Throughput, High-Availability, Lower-TCO Messaging.
Our next post covered 24×7 availability, reliability and lower TCO from this efficiency. Less hardware and fewer software processes to touch the data in transit between applications provides these benefits. For more, see the post Ultra Messaging: For 24×7 High Availability, Lower TCO, and Robust Reliability.
This post discusses how the same Ultra Messaging efficiency that provides performance, reliability, and lower TCO also provides great agility and near-linear scalability. And with today’s Big Data challenges, especially in the capital markets, efficiency is more prized than ever.
Our first post in this series (Ultra Messaging Is Also High-Throughput, High-Availability, Lower-TCO Messaging) covered, from a very high level, the performance benefits of highly-efficient messaging software by stressing that efficiency is the property of software that provides performance, whether you measure a single piece of data for “ultra-low latency”, or a large batch of data for throughput. Either way, ultimately, it’s all about extremely fast, lean, efficient execution. The way you choose to measure that performance is up to you, and depends on your needs.
But extremely fast, lean, efficient execution has other benefits for the customer besides performance. For example, the same Ultra Messaging efficiency that provides very high performance also provides the foundation for many of the key features of enterprise-quality software, such as true 24×7 high availability, lower total cost of ownership (TCO), and robust reliability. In the earlier post, we just touched on these topics, but here we will discuss them in a bit more detail.
From fuel-efficient cars, to energy-efficient homes and office buildings, to saving time by shaving while reading Twitter on your smart phone and also making sure your kids eat their breakfast, efficiency is on our minds more and more today. And for very good reason.
Efficiency is one of the most sought-after qualities of any product or service, because it means more for your money compared to the competition, and therefore, more overall buyer satisfaction. The product works, it works well, it works when you need it, it doesn’t quit unexpectedly or too soon, and you would buy it again in the future. (more…)
Many companies today must send streaming data across the globe, quickly, which often means use of a shared resource: a WAN. Bandwidth for most WANs is usually restricted to something like 100Mb/sec, or even 10Mb/sec, which is often much slower than the high-speed LANs connected to either side of the WAN. This is especially true in the capital markets where ultra low latency messaging is key.
A link speed mismatch like this can present a problem. Say you’re running a 10Gb Ethernet LAN in New York, and sending data to London and Singapore over a 100Mb WAN link. While your streaming market data has an aggregate rate well below 100Mb, the spikes are many multiples of that aggregate rate. Those spikes are where your problems can begin. (more…)