Data Mining, Visual Analytics, and The Human Component!

With all the massive amounts of data we are collecting from the Internet, well, it is just amazing the things we can do with it all. Of course, those concerned about privacy, well, you can understand why organizations like the Electronic Freedom Foundation is often fit to be tied. Still, think of all the good that can become of all this data? Let me explain.

You see, with the right use of visual analytics and various data mining strategies, we will be able to do nearly anything we need too. And, yes, I guess it goes without saying that I have a ton of thoughts on Visual Analytics of the Internet, Mobile Ad Hoc networking, and Social Networks along with some concepts for DARPAs plan for "crowd sourcing" innovation, it makes perfect sense to me, as each participant becomes basically a "neuron" and we use the natural neural network scheme.

What we need is a revolution in data mining visual analytics, so the other day I spent 20-minutes considering this and here are my thoughts. I propose an entirely new concept herein. Okay so let me explain my concept. But first let me briefly describe the bits and pieces of ideas and concepts I borrowed from to come up with this;

    There is an only UFO or Sci Fi tale I read, where the alien race said; "There is a whole new world waiting for you if you dare to take it,"
    Taking the "it" part of that line and calling "it" = "IT" as in Information Technologies.
    Next, combining that "IT" or "It entity" with that old Christian apocalyptic "mark of the beast" and the old computer system in Belgium 30-years ago claiming to be big enough to track every world transaction, also nick-named the beast.
    Then combining that concept with V. Bush's concept of "recording a life" or the later "life log theory" from Bell Labs.
    Then using the concept of the eRepublic, where government is nothing more than a networked website.
    Then considering the thought of Bill Gate's concepts in "the Road Ahead" where the digital nervous system of a corporation was completely and fully integrated.
    Combined with SAPs, and Oracles enterprise solutions
    Combined with Google's data bases
    Combined with the Pangaea Project for kids to collaborate in elementary school around the world and programming the AI computer, using a scheme designed by Carnegie Mellon to crowd source the teaching of an AI system. "eLearning Collaborative Networks like Quorum or Pangaea"
    Combined with IBMs newest mind map visualization recently in the news..
    Combined with these following thoughts of mine:

    My Book; "The Future of Truck Technologies," and 3D and 4D Transportation Computer Modeling; Page; 201.
    My Book; "Holographic Technologies," specifically; Data Visualization Schemes; Page 57 Chapter 5.
    My Article on 3D and 4D Mind Maps for Tracking and Analyzing.
    My Article on Mind Maps of the Future and Online style Think Tanks
    My Article on Stair Step Mentorship for Human Learning in the Future and Never Aging Societies.

Okay now let me explain the premise of my concept for Visual Analytics;

First, forget this whole idea of a 2D mind mapping concept or chart used to show links between terrorist players, cells, assets, acquaintances, etc., the way it is laid out currently - make it 3D, actually make it 4D and 5D where some layers can only be seen by a select few, and let's say a 6D level that can only be accessed by an AI super computer [why; because I don't trust humans, they can't be trusted, i.e. WikiLeak, leaker for instance].

Next ALL the data is stored within in the sphere. But to access the data on the outer side of the sphere, picture Earth's surface, the ball or sphere (with grids like a map of the globe) rolls around on a giant grid paper. When you want to look at a particular event, person, subject, or whatever, a particular point on the sphere's grid touches a corresponding point on the grid paper it rolls on, the grid paper it rolls on can wrap around and morph itself to the sphere or contour itself so the next corresponding piece of information on the surface can be accessed, rolling or spinning.

Picture a selectric typewriter ball on a shaft as a 2D model to consider this, now make it all 3D in your mind, and the paper molds around the sphere as it accesses, or in the case of a selectric typewriter it types. Now the Sphere is hollow inside containing layers, just like the earth, crust, mantel, and core. Information goes deep or across, every piece of information is connected, think about the earliest string theory models for this.

Great thing about my visualization concept is I believe all this math exists, even though in reality string theory is mostly bunk, but the math to get there makes this possible. As the information goes deep, think about the iPad touch screen, or the Microsoft restaurant "menu on a table" concept, or the depictions of Minority Reports, moving of the screens by way of motion gestures, I believe Lockheed also has this concept up and running for air-traffic control systems, prototype versions, perhaps the military is already using it, as it has massive applications for the net centric battlespace visualization too.

Okay so, some levels go through a frame-burst scenario taking you into another level, where the data generally stored at the almost infinite number of grid points and cross connected to every other is nothing more than a nucleus with additional data spinning around it. But the user cannot access all that information, without clearances, the AI system has access to all of it, while a sorting system is a series of search features within search features, with non-linked data also. You can't break into it; it's not connected to the users' interface at all, think of the hidden data as electrons unattached around the data. The data is known to exist but cannot be accessed that would be the 5D level, and 6D level no human may get too, but the data exists.

You know that surfer dude in Hawaii that came up with the "Grand Theory of the Universe" why not use his model for our visualization, in spherical form, again, the mathematics for all this already exists.

You see, what I need is a way to find people like me, I want to find these thinkers and innovators to take it all to the next level, and if the visualization is there, we can find; The Good Guys, Bad Guys, and the Future all at once. Why do I want a "Neural Network" visualization system in a sphere? It seems to me that this is how the brain does things, and what we are doing here is creating a Collective Brain, using each individual assigned to an "ever-expanding" unit of data, along a carrier or flow.

Remember when Microsoft Labs came out with that really cool way to travel through the Universe and look at all the celestial bodies along the way, using all the Hubble Pictures collected? It's kind of like that, you travel to the information, discover as you travel and it piques your curiosity as you go triggering your own brain waves, and splashing the users minds with chemical rewards as they go, as they discover more information, expanding their understanding as well, it just seems to me this is how it all works anyway.

Think of that old Sci Fiction concept where the Earth and our solar system are merely an atom of a chemical compound within a cell of the human body, all we can see is all the other compounds around us because everything is so small, thus, we cannot see the whole picture and what appears to be an entire universe would only be a few thousand cells close enough for us to see. And time itself is slow, as the electrons or planets moving around the atom appears to take a year to circle the nucleus instead of 10,000 times a second.

So, combining all these types of thoughts, this is how I envision how the future visualization tools would work.

Now then, using the whole concept of connecting the dots for information or even building an AI search feature scouring the system at speeds of terabytes a second, the AI computer can become the innovator, thanks to the user asking the question, and all the neurons (individual humans) with all their data putting in the information. You just need the best questions, you get instance answers.

Okay so, take this concept one step further; the AI super computer's operation is a "brain wave" and that brain wave is assigned a number, you can have as many brain waves, as the internet has IP addresses, with whatever scheme for that you choose. And your query can search the former queries too. The user's questions are as important as the data itself.

Thus, it helps us find the innovators, the question askers, once we know that, we have the opportunity for unlimited instant knowledge. Data visualization can take us there, and it removes all the fog of uncertainty, and answers most all the questions we could ever hope to ask, and comes up with its own questions as well. Does this make sense?

This is the type of visualization I need to faster access information, and I can solve all the problems, even the ones humans refuse to solve, or doom themselves to repeateth. That's my preliminary thought on this - may we start such a dialogue on the topic? If so, email me, and I hope you enjoyed today's dialogue?


Source: http://ezinearticles.com/?Data-Mining,-Visual-Analytics,-and-The-Human-Component!&id=4817019

The Need for Specialised Data Mining Techniques for Web 2.0

Web 2.0 is not exactly a new version of the Web, but rather a way to describe a new generation of interactive websites centred on the user. These are websites that offer

interactive information sharing, as well as collaboration - a case in point being wikis and blogs - and is now expanding to other areas as well. These new sites are the result of new technologies and new ideas and are on the cutting edge of Web development. Due to their novelty, they create a rather interesting challenge for data mining.

Data mining is simply a process of finding patterns in masses of data. There is such a vast plethora of information out there on the Web that it is necessary to use data mining tools to make sense of it. Traditional data mining techniques are not very effective when used on these new Web 2.0 sites because the user interface is so varied. Since Web 2.0 sites are created largely by user-supplied content, there is even more data to mine for valuable information. Having said that, the additional freedom in the format ensures that it is much more difficult to sift through the content to find what is usable.The data available is very valuable, so where there is a new platform, there must be new techniques developed for mining the data. The trick is that the data mining methods must themselves be flexible as the sites they are targeting are flexible. In the initial days of the World Wide Web, which was referred to as Web 1.0, data mining programs knew where to look for the desired information. Web 2.0 sites lack structure, meaning there is no single spot for the mining program to target. It must be able to scan and sift through all of the user-generated content to find what is needed. The upside is that there is a lot more data out there, which means more and more accurate results if the data can be properly utilized. The downside is that with all that data, if the selection criteria are not specific enough, the results will be meaningless. Too much of a good thing is definitely a bad thing. Wikis and blogs have been around long enough now that enough research has been carried out to understand them better. This research can now be used, in turn, to devise the best possible data mining methods. New algorithms are being developed that will allow data mining applications to analyse this data and return useful. Another problem is that there are many cul-de-sacs on the internet now, where groups of people share information freely, but only behind walls/barriers that keep it away from the genera results.

The main challenge in developing these algorithms does not lie with finding the data, because there is too much of it. The challenge is filtering out irrelevant data to get to the meaningful one. At this point none of the techniques are perfected. This makes Web 2.0 data mining an exciting and frustrating field, and yet another challenge in the never ending series of technological hurdles that have stemmed from the internet. There are numerous problems to overcome. One is the inability to rely on keywords, which used to be the best method to search. This does not allow for an understanding of context or sentiment associated with the keywords which can drastically vary the meaning of the keyword population. Social networking sites are a good example of this, where you can share information with everyone you know, but it is more difficult for that information to proliferate outside of those circles. This is good in terms of protecting privacy, but it does not add to the collective knowledge base and it can lead to a skewed understanding of public sentiment based on what social structures you have entry into. Attempts to use artificial intelligence have been less than successful because it is not adequately focused in its methodology. Data mining depends on the collection of data and sorting the results to create reports on the individual metrics that are the focus of interest. The size of the data sets are simply too large for traditional computational techniques to be able to tackle them. That is why a new answer needs to be found. Data mining is an important necessity for managing the backhaul of the internet. As Web 2.0 grows exponentially, it is increasingly hard to keep track of everything that is out there and summarize and synthesize it in a useful way. Data mining is necessary for companies to be able to really understand what customers like and want so that they can create products to meet these needs. In the increasingly aggressive global market, companies also need the reports resulting from data mining to remain competitive. If they are unable to keep track of the market and stay abreast of popular trends, they will not survive. The solution has to come from open source with options to scale databases depending on needs. There are companies that are now working on these ideas and are sharing the results with others to further improve them. So, just as open source and collective information sharing of Web 2.0 created these new data mining challenges, it will be the collective effort that solves the problems as well.

It is important to view this as a process of constant improvement, not one where an answer will be absolute for all time. Since its advent, the internet has changed quite significantly as well as the way users interact with it. Data mining will always be a critical part of corporate internet usage and its methods will continue to evolve just as the Web and its content does.

There is a huge incentive for creating better data mining solutions to tackle the complexities of Web 2.0. For this reason, several companies exist just for the purpose of analysing and creating solutions to the data mining problem. They find eager buyers for their applications in companies which are desperate for information on markets and potential customers. The companies in question do not simply want more data, they want better data. This requires a system that can classify and group data, and then make sense of the results.While the data mining process is expensive to start with, it is well worth for a retail company because it provides insight into the market and thus enables quick decisions.The speed at which a company which has insightful information on the marketplace can react to changes, gives it a huge advantage over the competition. Not only can the company react quickly, it is likely to steer itself in the right direction if its information is based on updated data.Advanced data mining will allow companies not only to make snap decisions, but also to plan long range strategies, based on the direction the marketplace is heading. Data mining brings the company closer to its customers. The real winners here, are the companies that have now discovered that they can make a living by improving the existing data mining techniques. They have filled a niche that was only created recently, which no one could have foreseen and have done quite a, good job at it.


Source: http://ezinearticles.com/?The-Need-for-Specialised-Data-Mining-Techniques-for-Web-2.0&id=7412130

What is Data Entry - Which Data Entry Services Are Popular?

Today, data entry business is the fastest growing businesses in the world whether we think so or not but it is fact. This data online entry is dynamic and is in regular changes.

Data form entries can be described as numeric, alpha numeric, text and form entry. Data entries services are very useful in business firms and organizations as there is a huge demand of entry.

Therefore, the outsourcing form entry business is flexible and is require detailed information and accuracy of ease of access. If you will outsource data entry requirements it helps to improve information management system whether you are running a small business or large business company because information is vital asset in this industry.

You can release from the difficulty of all your information processing requirements because of high quality, cost effective services provided by data entry companies which also helps you to focus on other business development processes which are also important.

Specially trained and skilled excel, word entry professional from offshore countries provide you excellent services with significant suggestions. There are several advantages of information entry outsourcing some majors are:

o Lowest Possible Data-Entry Cost
o Accurate and Fast delivery
o Access of specialized service
o Increased client satisfaction
o Savings manpower and training costs
o Focusing energy and workforce on your core business

Some of the most important services provided by outsourcing companies are described below:

Catalogs Entries
Directories Entries
Numeric Information
Textual Information
Data Capture and Data Collection
Image Information
Online Form Information
OCR/ICR Processing

By outsourcing data-entry services you can enjoy the convenience and security of work done by data entries companies. DataEntryOutsourcing is provide complete data online entry services and you can get more information about our data online entry company so visit our website.

Data entry outsourcing is an equal employment opportunity provider. We are dedicated to ensure the welfare of our employees. For more details data entry work, Contact us at: Data Entry


Source: http://ezinearticles.com/?What-is-Data-Entry---Which-Data-Entry-Services-Are-Popular?&id=3567541

Utilize Online Data Entry Services From India For Extended Profits

Numerous companies are seeking help for data entry services in order to better manage their company database. Many changes and innovations have taken place in this field that has greatly facilitated and accelerated often tedious and time consuming processes. There are a multitude of data entry companies offering useful solutions to businesses of all types. Hiring an online data entry company in India is an economical option. These outsourcing companies provide high quality work to businesses around the world.

Many firms have started passing their data entry work on to third party firms in an effort to save on overhead costs. The burden of employing a full time data entry associate on a salary basis and providing them with all the benefits of regular employees is immense, whether the work load is large of small for a particular month. Hiring a consultant will enable you to only incur costs when necessary.

Data is a very important part of any business. Therefore it is critical that it is handled with experienced hands and skills. Well managed data can be used in an endless number of ways to better plan and manage the undertakings of a business. As a part of data entry services, data conversion is gaining popularity. The companies are using the latest techniques and tools for designing data entry solutions. In order to make data useful for anyone, its conversion is important. Data that requires a longer duration of processing time should be outsourced for better efficiency of information flow.

Different types of businesses make use of these services. They primarily have an immense database to manage, for which to India is a cost effective option. For instance, pharmaceutical companies, educational institutes, law firms, E-Commerce sites, and others make use to data entry services for higher profits.

Outsourced data entry services have been very beneficial for companies by increasing sales, yet lowering labor expenses. They help in expanding the client base internationally, which enables global access to international customers with ease. Generally the data that is complex and takes longer processing times should be outsourced for better, timelier results. Some businesses can even take advantage of data conversion, document processing, and catalog development if they need it.

There are many advantages of outsourcing data entry services to developing countries like India. Here the workforce is cheap yet highly skilled. Thus, businesses can get better quality of work at relatively lower prices in comparison to companies in developed countries. There are a tremendous number of data entry service providers. One of these firms will be a great tool to take your business to the next level.



Source: http://ezinearticles.com/?Utilize-Online-Data-Entry-Services-From-India-For-Extended-Profits&id=1387446

Data Mining Software - Discover Software Modernization

Data mining software is usually an application that one uses and covers mostly with one's knowledge in the discovery of software modernization. Mining data software involves the understanding of the software artifacts that exist and the mining data tools. This process has very close relations with reverse engineering. The knowledge that one gains from studying data software that exists is usually presented in forms of models and by doing these queries one can be in a position to make his personal data mining software. With the knowledge that someone gains it must be applicable and one must also know the mining data tools that are suppose to be used apart from the soft wares. One can be able to know very widely about the mining data tools that are there in mining data software by doing computer science as a course. Computer science covers widely on what are the procedures, steps of mining data software and how can use the mining data tools.

This software is mostly used in making of databases schemes. Making of databases is not as easy as many would think it requires one to have some knowledge about computer engineering and the basic concepts of computers.;This software is mostly used in data crawling because it can be in a position to store data and one can be able to retrieve the data when needed.

The softwares are not that cheap they come in different varieties and it will depend on which information or the database on which one is coming up with.

Data mining software are usually in different levels there is the data level, design level, application level, architectural level, call graph level and program level it will depend on which level one is covering and this come together with mining data tools.

Data software's have increased rapidly through the introduction of computers and ERP definition. Computers hackers have been able to get the softwares at a very low price and this has made data mining to become very easy and quick to use in the shops and supermarkets and also government institutions. One cannot do data crawling without having the basic knowledge about data mining soft wares because soft wares are the programmes that are usually installed into the computer and without the programmes then no data can be processed.

There are a lot of challenges that come with the use of the mining soft ware. One can easily crush the software he is using or the softwares can easily break they are normally sold on CDS one can easily break it or loose it.

High chances of losing the data that someone is coming up with is very high because computers easily crash due to some difficulties that they experience or a virus can easily crush the computer.

Mining software take a very large space and in most of the computers. The reason behind this is because, data crawling use graphics. Graphics usually occupy a lot of space in terms of the size of the local disk. One is suppose to look for a computer that has very good memory. Data crawling is something that needs to be updated each and every time something appears along the way.


Source: http://ezinearticles.com/?Data-Mining-Software---Discover-Software-Modernization&id=5054991

Things You Should Know about Data Mining or Data Capturing

The World Wide Web is a portal containing billions of quality information, spanning resources from around the globe. Through the years, the internet has developed into a competitive business environment which offers advertising, promotions, sales and marketing innovations that has rapidly created a following with most websites, and gave birth to online business transactions and unprecedented financial growth.

Data mining comes into the picture in quite an obscure procedure. Most companies utilize data entry level workers to edit or create listings for the items they promote or sell online. Data mining is that early stage prior to the data entry work which utilizes available resources online to gather bits and pieces of information relevant to the business or website they are categorizing.

In a certain point of view, data mining holds a great deal of importance, as the primary keeper of the quality of the items being listed by the data entry personnel as filtered through the stages under data mining and data capturing.

As mentioned earlier, data mining is a very obscure procedure. The reason for my saying this is because of the fact that certain restrictions or policies are enforced by websites or business institutions particularly on the quality of data capturing, which may seem too time-consuming, meticulous and stringent.

These methodologies are but without explanation as well. As only the most qualified resources bearing the most relevant information can be posted online. Many data mining personnel can only produce satisfactory work on the data entry levels, after enhancing the quality of output from the data mining or data capturing stage.

Data mining includes two common strategies. The first one would be a strategy based on manual labor and data checking, with the use of online or local manual tools and scripts to gather the right information. The second would be through the use of web crawlers or robots to perform the task of checking for information on various websites automatically. The second stage offers a faster method for gathering and listing information.

But often-times the procedure spit out very garbled data, often confusing personnel more than helping.

Data mining is a highly exhaustive activity, often expending more effort, time and money than other types of work. Leveling them out, local data mining is a sure fire method to gain rapid listings of information, as collected by the information miners.


Source: http://ezinearticles.com/?Things-You-Should-Know-about-Data-Mining-or-Data-Capturing&id=256125

Using Forensic Social Media Data Mining to Discover Work Comp Fraud

As an employer, most of your employee based work comp claims are completely legitimate and should be handled in the best interest of the injured employee. Unfortunately, some are also fraudulent, causing increasing costs on baseless claims. Historically, it's been challenging to contest some of these claims, though recently a new road has opened, allowing enlightened employers to more rapidly travel this road to a truthful outcome.

Employers should now consider the usefulness of Facebook, YouTube, LinkedIn and other social media sites which can contain posts negating the claims of allegedly injured workers participating in activities that are beyond the restrictions placed by the treating physician. These posts can happen on any given day, clearly elucidating a fraudulent claim. For example, let's say that an employee is out of work based on a "work comp" (workers compensation) claim which has restrictions, yet they post links, discussions, comments and photos that are clearly incriminating. This is an area in which companies should be mining regularly to protect your company against out of scope workers compensation claims.

Recently, a transportation attorney based in central Pennsylvania was a guest speaker at an insurance transportation web seminar. His topic was the aggressive defense of trucking lawsuits and he elaborated extensively on an example of forensic social media investigation to assist in the aggressive defense of frivolous lawsuits. I recall that the metrics were impressive; one example noted that a $250,000 claim which was reduced to $2,500 when forensic social media data mining found New Years Eve photos proving the claimant's mobility to be much greater than stipulated in the law suit.

Social media offers a surprising if not inadvertent glimpse into the nuances of the lifestyles of anyone using it, and in certain cases, it also offers important evidence into the veracity of work comp and work comp lawsuit based claims. Employers should be aware of this avenue and investigate accordingly.

The D'Camera Group http://www.dcameragroup.com partners with businesses, creating long range plans to manage their Total Cost of Risk. D'Camera Group's proprietary approach bridges the gap between the client and the insurance marketplace through a series of engagements in which we Discover, Design and Implement a Risk Reduction Plan™. Our specialized business insurance services include risk discovery and assessment, risk reduction strategies, insurance loss management, workers compensation, experience mod factors and improving marketplace competitiveness.


Source: http://ezinearticles.com/?Using-Forensic-Social-Media-Data-Mining-to-Discover-Work-Comp-Fraud&id=5027122