In an AI project, going live is only the start. Learn how to structure data, integration, and operational roles to maintain and monitor your AI effectively without unnecessarily over-hiring.
In a "data artificial intelligence" project, deploying to production does not mark the end of the work. Data changes, connected tools evolve, and users encounter edge cases the prototype had never seen. The question then becomes very practical: who detects a degradation, who fixes it, and who decides whether the service can continue running?
For an SME or a scale-up, several responsibilities need to be covered, not necessarily a full team hired. Here is how to choose the right profiles, distribute tasks between employees and external providers, and organize maintenance tailored to your business.
A document assistant, a forecasting model, and an automation connected to your CRM do not require the same team. Before hiring, identify what actually needs to be maintained.
An assistant based on an external model primarily requires integration, response evaluation, and document management skills. The vendor operates the model, but your company remains responsible for permissions, the documents used, and the impact of the responses on its processes.
A predictive model trained on your data, such as one for forecasting demand, also requires monitoring its performance and organizing potential retraining. A data scientist then becomes more relevant.
An automation acting within your software demands special attention to connectors, permissions, and runtime errors. An accurate response is not enough if the tool creates the same order twice.
This distinction prevents hiring a model specialist to solve a problem that is primarily about software development or data quality.
One person can take on multiple functions. However, no function should be left without an identified owner.
Function | Primary Responsibility | Profile That Can Fill It |
|---|---|---|
Business Ownership | Define expected outcomes, acceptable errors, and priorities | Operational manager or product owner |
Data Management | Ensure reliable, updated sources accessible to the right people | Business champion, data analyst, or data engineer |
AI Engineering & Integration | Maintain pipelines, connections, and technical evaluations | Experienced developer or AI engineer |
Operations & Security | Monitor the service, manage access, and organize incident response | Technical team, DevOps profile, or service provider |
The business owner validates the system's utility. The data manager knows the authoritative sources. The engineer knows how to modify the solution and restore service. Operations ensures alerts reach someone capable of taking action.
In a "data artificial intelligence" setup, a single individual can combine several of these functions, but they must have the necessary time and access. A responsibility written on a document is not enough.
Not necessarily. If your assistant uses an existing model and consults your documents, early needs often revolve around data pipelines, development, and business acceptance testing. A data engineer or a developer proficient in these areas may be more useful than a model training specialist.
A data scientist becomes more relevant when you build a custom statistical model, analyze its errors, or need to decide when to retrain it. MLOps—the technical organization for deploying and monitoring models in production—becomes essential when these operations are recurring.
Hire for the recurring work that needs to be done, not to check off every job title in a data team.
The right choice depends on the criticality of the service, the volume of recurring work, and your ability to maintain control over the project.
Setup | Relevant When… | Watch Out For |
|---|---|---|
Primarily In-house | AI supports a core activity and requires frequent updates | Plan for coverage during absences or departures |
Primarily Outsourced | Scope is well-defined and technical needs are intermittent | Frame response times and reversibility |
Hybrid | The company wants to retain business decisions while pooling technical expertise | Clearly define everyone's responsibilities |
For a small organization, the hybrid model often serves as a sensible starting point: an internal business lead, a data owner, and a technical partner. This does not guarantee sufficient capacity on its own. An occasionally used tool and an indispensable all-day service do not demand the same service-level agreements.
Outsourcing the "data artificial intelligence" scope should not mean outsourcing business decisions. Your company must continue to define acceptable use cases and validate outputs.
If you are still hesitating between off-the-shelf software, assembling components, or custom development, start by choosing a solution adapted to your processes. The chosen architecture will determine part of the required maintenance skills.
Maintenance limited to checking that the server is responding misses the point. The service can be up while generating useless responses.
The team must know the origin of the data, its update frequency, and who is authorized to correct it. For a document assistant, this includes indexing rules and the removal of outdated content.
In a RAG architecture, where the AI retrieves information from your sources before answering, adding a document does not necessarily train the model. The key is ensuring this document is retrieved at the right time and only by authorized users.
Maintaining an artificial intelligence system therefore relies on data management as rigorous as that of code. An interrupted synchronization can degrade responses without triggering any visible error for the user.
For sensitive content, also define what information can be transmitted to the provider and what must remain excluded. These rules extend the measures required to protect your internal data in an AI chat.
Build a set of test cases validated by business users: frequent requests, ambiguous situations, and prompts the tool should refuse to answer. For automation, test executed actions and partial failures as well.
This benchmark is used to compare versions before any modification to the model, prompt, or source documents. It should evolve alongside encountered incidents, without becoming an artificial collection of cases disconnected from real life.
Monitoring must separate three dimensions: output quality, technical reliability, and business value. Measure, for example, required corrections, execution failures, and actual time saved after review.
The NIST AI Risk Management Framework provides a useful benchmark for structuring governance, measurement, and risk management. It does not replace your business criteria and does not constitute, on its own, proof of compliance.
Monitoring must cover errors, latency, and usage costs. Depending on the system, add data freshness, missing documents, or actions denied by connected software.
Logs should allow investigation without unnecessarily storing personal data or confidential content. Define access permissions and retention periods. The CNIL Personal Data Security Guide provides guidance on logging, authorizations, and incident management.
The team must also version code, prompts, and configurations. Plan for rollbacks to a validated version whenever technically possible. With an external API, you do not always control provider model updates: a degraded mode or suspending certain features may be necessary.
Consider an internal assistant answering questions about sales procedures. After an update, it continues citing an outdated pricing rule.
The business lead confirms the error and its severity. The data manager checks which document version is authoritative. The engineer verifies indexing and document retrieval, then fixes the issue. The business lead validates testing before full restoration of service.
This example shows why a "data artificial intelligence" project cannot rely solely on the person who built the prototype. Resolving an issue also requires a decision on the correct information and on the business impact of the incident.
Before the first incident occurs, agree on four things: who receives the alert, who can disable the affected feature, who intervenes technically, and who authorizes its reactivation. Also clarify coverage hours. Business-hours maintenance is not the same as 24/7 on-call support.
The first month should be used to establish sustainable operations rather than multiplying monitoring tools. The following schedule is an example to tailor to your service's criticality.
Week 1: Assign responsibilities. Appoint leads, inventory access, and document external dependencies.
Week 2: Establish a baseline. Measure quality on business cases, response times, and usage costs before setting targets.
Week 3: Run an incident simulation. Simulate an unavailable source or a failing connector and verify the escalation path.
Week 4: Prioritize action items. Review recurring errors, user requests, and the required maintenance capacity.
By the end of this cycle, you should know where code, configurations, and recovery procedures live. You should also know which areas your team cannot handle alone.
Reviewing the "data artificial intelligence" scope should lead to actionable decisions: fixing a pipeline, strengthening tests, limiting a use case, or bringing in additional expertise. A dashboard with no owner or associated action provides little protection.
Scale the team when recurring work exceeds its available capacity or when a missing skill jeopardizes service reliability. The number of users alone is not a decisive factor: a rarely used automation can trigger highly sensitive operations.
If incidents mainly stem from data sources and synchronizations, focus on data engineering. If integrations break or deployments are fragile, reinforce development and operations. If a predictive model loses accuracy, bring in data science expertise.
The budget must distinguish between usage costs, corrective maintenance, and new features. Then compare total cost of ownership against the measured business value, factoring in required human oversight.
Finally, assess reliance on key individuals. Usable documentation and a backup person capable of stepping in may be more urgent than a new specialized hire. This continuity matters just as much for an internal employee as it does for a service provider.
Does an SME need to create a data department to maintain its AI? No. It must first cover business ownership, data, integration, and operations responsibilities. These functions can be split between existing team members and a partner, provided there is real capacity to act.
Does an AI provider handle all maintenance? No. They manage the components covered by their service offering. Your organization remains responsible for its data, integrations, permissions, and validating use cases.
Do all models need to be retrained regularly? No. An assistant using an external model primarily requires document updates, integration fixes, and testing. For an internally trained model, retraining decisions should be driven by observed performance degradation.
What is the first deliverable to request for a data artificial intelligence project? Request a responsibility matrix along with quality criteria and an incident procedure. It should clarify who decides, who fixes issues, and how to verify return to acceptable operations.
Before hiring, clarify the components to maintain, existing skills, and business risks. Impulse Lab offers AI opportunity audits, custom development, and adoption training. You can talk with Impulse Lab about your project to scope the technical and organizational needs of your solution.
Our team of experts will respond promptly to understand your needs and recommend the best solution.
Got questions? We've got answers.
Leonard
Co-founder
