Rasqal RDF Query Library - Release Notes

Rasqal 0.9.21 Changes

The main changes in this release are to add support for features of SPARQL 1.1 Query W3C working draft of 14 October 2010:

Fixed Issues:

Query Engine changes

Support executing new SPARQL 1.1 draft function isNUMERIC().

Support executing built-in aggregate expressions:

with and without DISTINCT across the arguments.

Support executing grouping of results with GROUP BY (or implicitly if aggregate function was given) and HAVING to filter groups.

LAQRS Query Language changes

Explain more carefully that EXPLAIN and COALESCE() are LAQRS only and remain experimental.

SPARQL Query Language changes

Separate SPARQL 1.0 and SPARQL 1.1 draft language names giving query languages sparql (with alias sparql10 for SPARQL 1.0), sparql11 for SPARQL 1.1 and laqrs for experiments. The sparql name without a version currently points to SPARQL 1.0 but over time will advance to be SPARQL 1.1

Added support for parsing (not executing) new SPARQL 1.1 draft tokens: BINDINGS for variable bindings, UNDEF, SERVICE and MINUS.

Added support for parsing and executing new SPARQL 1.1 draft isNUMERIC() expression.

Added support for parsing (not executing) new SPARQL 1.1 draft sub-SELECT queries.

Support for parsing and executing built-in (not user) aggregate expressions: AVG(expr), COUNT(*) and COUNT(expr), GROUP_CONCAT(expr) with optional ;separator = 'string', MAX(expr), MIN(expr), SAMPLE(expr) and SUM(expr) with and without DISTINCT across the arguments.

Added support for executing grouping of results with GROUP BY (or implicitly if aggregate function was given) and HAVING to filter groups.

Bindings class changes

Added a class to support for SPARQL 1.1 draft BINDINGS in a query specifying a set of variable bindings and a sequence of rows of values to bind to them. Intended for applying to a SPARQL 1.1 draft query SERVICE call.

Added bindings class rasqal_bindings with constructor rasqal_new_bindings, destructor rasqal_free_bindings() and method rasqal_bindings_print()

Added accessor methods for getting bindings variables and rows rasqal_query_get_bindings_variables_sequence(), rasqal_query_get_bindings_variable(), rasqal_query_get_bindings_rows_sequence() and rasqal_query_get_bindings_row()

Data Graph class changes

Turn a data graph into a reference-counted class and add support for specifying the format of a data graph for parsing after retrieval.

typedef rasqal_data_graph gains new optional format_type, format_name and format_uri fields for describing the format of the graph an optional uri field for base URI and a usage field.

Added constructor rasqal_new_data_graph_from_uri() with format_type, format_name and format_uri args deprecating rasqal_new_data_graph() without them, which always guessed the format.

Added constructor rasqal_new_data_graph_from_iostream() taking a raptor_iostream arg to allow getting data graphs from non-URIs which could be strings, memory etc.

Added copy constructor rasqal_new_data_graph_from_data_graph() for data graphs.

Expression and literal class changes

Added support for SPARQL 1.1 draft isNUMERIC() expressions. The rasqal_op gains a new RASQAL_EXPR_ISNUMERIC value.

Added rasqal_expression_op_label() to return the label for an expression operation.

Added rasqal_expression_compare() to compare literals.

Added rasqal_expression_is_aggregate() to test if an expression is an aggregate one - has an aggregate operation or user-defined function.

Added rasqal_literal_type_label() to return the label for a literal type.

Graph pattern class changes

rasqal_graph_pattern_operator enumeration gains new values for SPARQL 1.1 draft features: RASQAL_GRAPH_PATTERN_OPERATOR_SELECT for sub-SELECT, RASQAL_GRAPH_PATTERN_OPERATOR_SERVICE for service binding and RASQAL_GRAPH_PATTERN_OPERATOR_MINUS for minus operation between graph patterns.

Added new constructor rasqal_new_single_graph_pattern() for use with service and minus graph patterns.

Added rasqal_graph_pattern_get_variable() method to get the variable for SPARQL 1.1 draft BIND and LAQRS LET graph patterns.

Added rasqal_graph_pattern_get_service() method to get the service object for SPARQL 1.1 draft SERVICE graph pattern.

Query class changes

Added rasqal_query_add_data_graph2() to add a previously constructed data graph to a query deprecating rasqal_query_add_data_graph() that did construct and add in one go.

Added rasqal_query_add_data_graphs() to add a sequence of data graphs to a query.

Added rasqal_query_add_data_graph_from_iostream() using new rasqal_new_data_graph_from_iostream().

Note in the docs for rasqal_query_set_limit() and rasqal_query_set_offset() that with the new query engine approach, you cannot set the limit or offset until the query is prepared which is after parsing.

Query results class changes

Add a formatter to write a variable bindings query results in Turtle format with the schema used for the W3C DAWG and SPARQL Working group test cases.

rasqal_query_results_formats_check2() added with new flags arg for asking for a format to read or write, deprecating rasqal_query_results_formats_check() without it.

Row class changes

The rasqal_row gains a group_id field for returning group boundaries. Change of groups should be detected by a change in IDs between rows.

Service class changes

Added a rasqal_service class for executing remote queries against a SPARQL HTTP protocol endpoint and getting back a query results set. The data graphs attached get turned into default-graph-uri and named-graph-uri URI parameters to the request.

Added constructor rasqal_new_service(), destructor rasqal_free_service() and methods rasqal_service_set_www() to set the WWW object rasqal_service_set_format() to set the accept request header and rasqal_service_execute() to execute the query returning the results.

Triples source class changes

Add a way for a triples source factory to return meaningful errors by adding a new init_triples_source factory method to the rasqal_triples_source_factory that takes a rasqal_triples_error_handler callback. The RASQAL_TRIPLES_SOURCE_FACTORY_MAX_VERSION is now 2 and ranges 1 to 2 are supported.

Add a way for a triples source to indicate support for optional features by adding a support_feature method toe the rasqal_triples_source structure. The RASQAL_TRIPLES_SOURCE_MAX_VERSION is now 2 and ranges 1 to 2 are supported.

Added enum rasqal_triples_source_feature with value RASQAL_TRIPLES_SOURCE_FEATURE_IOSTREAM_DATA_GRAPH to indicate a triple source supports reading data graphs from a raptor_iostream.

Variable class changes

Now a reference-counted class.

Deprecated rasqal_new_variable() and rasqal_new_variable_typed() for rasqal_variables_table_add().

Internal changes

All destructors accept NULL objects and do nothing.

Add a Raptor V2-style raptor_syntax_description data model to Rasqal and use it to initialise query languages. It is not currently exposed in the API and may not be until Rasqal requires Raptor V2 when it can share code.

Added an internal rasqal_bindings structure for SPARQL 1.1 BINDINGS.

Add an internal query results modifiers structure rasqsal_solution_modifier class for use in storing group by, having, limit, offset information which together SPARQL 1.1 documentationc calls modifiers.

Added rowsources for grouping, executing aggregated queries across groups, having filters per group. Modified the projection rowsource to handle aggregation.

roqet(1) utility changes

Added support for serializing to graphs or bindings with the same Raptor serializer / Rasqal result formatter name such as for example 'json'.

Added -F NAME option for specifying the data graph format; otherwise Raptor uses the guessing mechanism.

Added support for reading a data graph from standard input (stdin) when the -D / --data , -G / --name or -s / --source options are used with value '-'.

Added -p / --protocol SERVICE-URI option to execute a remote query at a SPARQL HTTP protocol URI. This is used with -e query string to specify the query to execute remotely.

Configuration and build changes

Use AC_LANG_SOURCE for gcc flag test.

Other changes

The srxread and srxwrite example programs now build when Rasqal is configured and built with raptor V2.

Rasqal 0.9.20 Changes

The main changes in this release are:

Fixed Issues:

Query API changes

Moved generate blank node ID functionality to world class. Deprecated rasqal_query_set_default_generate_bnodeid_parameters() for rasqal_world_set_default_generate_bnodeid_parameters(). Deprecated rasqal_query_set_generate_bnodeid_handler() for new function (with different handler signature) rasqal_world_set_generate_bnodeid_handler().

Added rasqal_query_get_having_conditions_sequence() and rasqal_query_get_having_condition() for getting access to the HAVING expression list in a query.

Expression and literal class changes

Added new literal type RASQAL_LITERAL_INTEGER_SUBTYPE to let subtypes of xsd:integer work in data and expression evaluation.

Add support for SPARQL 1.1 draft FILTER expression operators including execution in the expression evaluation and writing the expression as SPARQL:

New rasqal_op enum SPARQL 1.1 expression
RASQAL_EXPR_IF IF(condition expression, expr if true, expr if false)
RASQAL_EXPR_URI URI(uri string)
RASQAL_EXPR_IRI IRI(iri string)
RASQAL_EXPR_STRLANG STRLANG(literal string, language string)
RASQAL_EXPR_STRDT STRDT(literal string, datatype URI)
RASQAL_EXPR_BNODE BNODE() and BNODE(string)
RASQAL_EXPR_IN Expr IN ( list )
RASQAL_EXPR_NOT_IN Expr NOT IN ( list )

Added expression evaluation and expression formatting support for the above new (non-aggregate) expression operations in: rasqal_expression_clear(), rasqal_expression_visit(), rasqal_expression_write(), rasqal_expression_print(), rasqal_expression_is_constant() and rasqal_query_write_sparql_expression().

Added rasqal_op enumeration values for built-in aggregate functions RASQAL_EXPR_GROUP_CONCAT and RASQAL_EXPR_SAMPLE but no expression execution.

Added new bitflags rasqal_expression_flags for rasqal_expression flags with initial bitflags RASQAL_EXPR_FLAG_DISTINCT for distinct (aggregate functions) and RASQAL_EXPR_FLAG_AGGREGATE for aggregate functions.

Added rasqal_new_function_expression2() for functions with parameters and optional bit flags deprecating rasqal_new_function_expression(). This is not capable of adding a user extension aggregate function.

Added new rasqal_new_aggregate_function_expression() constructor for making a 1-arg built-in aggregate functions. This is not capable of adding a user extension aggregate function.

The never used RASQAL_EXPR_GROUP_COND_ASC and RASQAL_EXPR_GROUP_COND_DESC are marked as obsolete.

Updated documentation for rasqal_expression_visit(), rasqal_graph_pattern_visit_fn and rasqal_expression_visit_fn typedef to match what the code actually does.

Added rasqal_new_group_concat_expression() for building SPARQL GROUP_CONCAT() expressions with flags, non-empty list of expressions and optional literal separator.

SPARQL query language changes

Support latest SPARQL 1.1 Query W3C working draft of 1 June 2010 syntax such as:

For SPARQL 1.1 Query, allow DISTINCT optionally before extension function expressions with new flag RASQAL_EXPR_FLAG_DISTINCT.

Added draft SPARQL 1.1 Query IF(), URI(), IRI(), STRLANG(), STRDT() and BNODE() (with optional expression arg) built-in expressions.

Added draft SPARQL 1.1 Query IN() and NOT IN() operators.

Adjust draft SPARQL 1.1 Query select expression syntax to latest version: '(' Expr 'AS' Var ')' with required braces.

Added draft SPARQL 1.1 Query aggregate functions GROUP_CONCAT() and SAMPLE() with new expression enum RASQAL_EXPR_GROUP_CONCAT and RASQAL_EXPR_SAMPLE as well as formatting to SPARQL and placeholder expression handling implementation - no aggregate expression evaluation. Supports the optional ; SEPARATOR = "string" syntax for GROUP_CONCAT().

Add support for draft SPARQL 1.1 Query syntax GROUP BY a list of select expressions and HAVING with a list of expressions.

Add partial support for draft SPARQL 1.1 Query extension (aggregate) functions with DISTINCT and parameters.

Support latest SPARQL 1.1 Update W3C working draft of 1 June 2010 syntax such as:

Not all cases may be covered yet since the grammar is hard to implement and may be ambiguous.

LAQRS syntax support

Start giving deprecated messages for LAQRS SELECT ... AS varname (bare variable name) instead of the draft SPARQL 1.1 Query SELECT ... AS ?varname with $ or ? symbol.

Query engine changes

Make query result limit and offset work when no ORDER BY is part of the query (yes - this is a well-defined count of under-defined actual results) with the original query engine. Related to Issue#0000352.

Add datatype support for xsd:date for value normalization and comparisons.

Handle when results are sorted before used to build triples via CONSTRUCT, which has no practical use since triples are not ordered and is less efficient since you have to read all the results into memory. Anyway, it works now.

Query results formatter class changes

Added an HTML Table results format (name 'table') by Nicholas J Humfrey using patch from Issue#0000360.

A NULL base_uri is valid in rasqal_query_results_formatter_write()

World class changes

Moved generate blank node ID functionality to world class, deprecating the query class methods. Added rasqal_world_set_default_generate_bnodeid_parameters(), rasqal_world_set_generate_bnodeid_handler() with new handler typedef rasqal_generate_bnodeid_handler2.

Other changes

Made all rasqal destructors simply return when given a NULL object pointer. They mostly did this already.

Updated rasqal.spec.in based on fedora spec (not tested)

Replace strncpy() with memcpy() since the the strncpy() zero-checking and zero-filling semantics are never needed.

Minimum raptor version is now 1.4.19 so that the raptor_world typedef is always available.

Rasqal 0.9.19 Changes

The only change to this release is to fix the pkg-config rasqal.pc file to to restore the dependency on raptor which was accidently deleted. Lesson learnt: do not commit code after midnight local time.

Rasqal 0.9.18 Changes

The main changes in this release are:

Fixed Issues:

Query API Changes

A user data void* user_data field was added to the rasqal_variable structure to allow application code to store additional information.

Added a new rasqal_query_verb enum value for SPARQL 1.1 (draft) Update: RASQAL_QUERY_VERB_UPDATE which indicates that a sequence of update operations are available in the query structure, rather than a query (SELECT, ASK, DESCRIBE).

Added a rasqal_update_operation class for holding an updates that adds and/or removes triples from a graph (with possible conditional where) or does a graph administration operation on a grpah.

Added an rasqal_update_type enum for the types of graph update - clear, drop, delete, insert and, load.

Added rasqal_update_type_label() to get a label for a rasqal-update_type.

  const char* rasqal_update_type_label(rasqal_update_type type);

Added an rasqal_update_flags enum for capturing update operations flags such as SPARQL 1.1 Update SILENT for silent graph administration operations and DATA for bulk insert/delete data, rather than graph pattern templates with variables.

Added rasqal_query_get_update_operations_sequence() to get the sequence of updates from a query. Added rasqal_query_get_update_operation() get an individual update operation from a query via an index.

  raptor_sequence* rasqal_query_get_update_operations_sequence(rasqal_query* query);

  rasqal_update_operation* rasqal_query_get_update_operation(rasqal_query* query,
    int idx);

Query Results Class Changes

rasqal_new_query_results() added to public API to construct a standalone query result set without being the result of a query execution. (The destructor is already present). Added method rasqal_query_results_add_row() to add a row to a query result set.

  rasqal_query_results* rasqal_new_query_results(rasqal_world* world,
     rasqal_query* query, rasqal_query_results_type type,
     rasqal_variables_table* vars_table);

  int rasqal_query_results_add_row(rasqal_query_results* query_results,
     rasqal_row* row);

rasqal_query_results_type enum added to public API for use with rasqal_new_query_results().

Query Results Formatter Class Changes

Registered URIs for the CSV and TSV query result formats and fixed the mime types for those as well as the JSON format as reported in issues Issue #0000345 and Issue #0000323.

Added rasqal_new_query_results_formatter2() with format name, mime type and URI args deprecating rasqal_new_query_results_formatter() and rasqal_new_query_results_formatter_by_mime_type().

  rasqal_query_results_formatter* rasqal_new_query_results_formatter2(rasqal_world* world,
    const char *name, const char *mime_type, raptor_uri* format_uri);

Added rasqal_query_results_write2() and rasqal_query_results_read2() with both name and mime_type args deprecating rasqal_query_results_write() and rasqal_query_results_read() respectively.

  int rasqal_query_results_write2(raptor_iostream *iostr,
    rasqal_query_results *results, const char* name, const char* mime_type,
    raptor_uri *format_uri, raptor_uri *base_uri);

  int rasqal_query_results_read2(raptor_iostream *iostr,
    rasqal_query_results *results, const char* name, const char* mime_type,
    raptor_uri *format_uri, raptor_uri *base_uri);

Query Result Row Class

Export the query result row class to the public API to allow creating standalone query results set without a query.

Added rasqal_new_row_for_size(), rasqal_free_row() to construct and destruct a row and rasqal_row_set_value_at() to set row literal values.

  rasqal_row* rasqal_new_row_for_size(rasqal_world* world, int size);

  void rasqal_free_row(rasqal_row* row);

  int rasqal_row_set_value_at(rasqal_row* row, int offset,
    rasqal_literal* value);

Variables Table Class

Added variables table class rasqal_variables_table to the public API to allow use in creating a standalone query results.

Added rasqal_new_variables_table() and rasqal_free_variables_table() to construct and destroy a variables table. Added method rasqal_variables_table_add() to add a variable to the variables table.

  rasqal_variables_table* rasqal_new_variables_table(rasqal_world* world);

  void rasqal_free_variables_table(rasqal_variables_table* vt);

  rasqal_variable* rasqal_variables_table_add(rasqal_variables_table* vt,
    rasqal_variable_type type, const unsigned char *name,
    rasqal_literal *value);

SPARQL Changes

Added syntax-only and API support (no execution) for the SPARQL 1.1 Update W3C Working Draft of 2010-01-26 syntax. These are only available when using the 'laqrs' query language name, until SPARQL 1.1 is stable.

Added new lexer support for keywords: WITH, CLEAR, CREATE, SILENT, DATA, DROP, LOAD

Added CLEAR and CLEAR GRAPH uri

Added DROP GRAPH uri

Added INSERT DATA { triples }

Added DELETE templates to delete multiple subgraphs.

Added DELETE DATA { triples }

Added LOAD uri and LOAD uri INTO graph-uri

Added WITH uri DELETE { ... } INSERT { ... } WHERE ... update form since other combinations are ambiguous.

NOTE: The older LAQRS experimental INSERT and DELETE formats are supported but deprecated.

Query Engine changes

Nobody reported it, but decimal division was totally broken. Fixed it to do division and not add!

Other Changes

Multiple code style updates and code file reorganizing.

Link roqet directly with -lraptor to get raptor symbols used linked directly, to enable alternate library linking approaches to work. Based on the request in Fixes Issue#0000347 and Fedora bug 564885.

Added checks for the public API methods to ensure NULL pointers are not accepted or used.

Added example code for constructing a query result set without a query

Updated configure and the build system to use silent rules for the maintainer (by default), or when --enable-silent-rules is passed to configure. This feature requires building with automake 1.11 which requires autoconf 2.62 or newer when building from GIT.

autogen.sh script was updated to enforce the autotools versions above.

The rasqal.pc file now uses the Requires.private field for raptor; it should only be in the link line with
pkg-config --static ... is used.

Enforce that only raptor versions <1.9.0 are supported. No Raptor2 or Raptor2 alpha support yet.

Rasqal 0.9.17 Changes

WARNING: ABI AND API CHANGED IN THIS RELEASE.

This means: functions and structs were removed, modified and added.

Consequently the shared library major soname version has changed from 1 to 2.

The main change in this release is to add a new query engine and to enable it as the default query engine. It supports more of the SPARQL query algebra such as groups and optionals. There were many other changes and fixes.

Fixed Issues: (most of them in query engine 2)

Rasqal world API changes

Added methods for setting of an already-opened raptor library to pass into rasqal, rather than it creating one itself. rasqal_world_open() can then be called to do the actual initialisation:

  int rasqal_world_open(rasqal_world* world);

  void rasqal_world_set_raptor(rasqal_world* world,
                               raptor_world* raptor_world_ptr);

  raptor_world *rasqal_world_get_raptor(rasqal_world* world);

Exported new C defines for the library versions: RASQAL_VERSION in 6-digit integer MMNNRR format (Major, miNor, Release), RASQAL_VERSION_STRING in string "MM.NN.RR" format, RASQAL_VERSION_MAJOR for integer major version number, RASQAL_VERSION_MINOR for integer minor version number and RASQAL_VERSION_RELEASE for integer release number.

API CHANGE: rasqal_features_enumerate() gains an initial rasqal_world argument to become a method of the world class:

  int rasqal_features_enumerate(rasqal_world* world,
    const rasqal_feature feature, const char **name,
    raptor_uri **uri, const char **label);

API CHANGE: rasqal_feature_from_uri() gains an initial rasqal_world argument to become a method of the world class:

  rasqal_feature rasqal_feature_from_uri(rasqal_world* world,
    raptor_uri *uri);

Data Graph API changes

API CHANGE: The rasqal_data_graph structure gains a new world reference pointer field.

API CHANGE: rasqal_new_data_graph() gains an initial rasqal_world argument:

  rasqal_data_graph* rasqal_new_data_graph(rasqal_world* world,
    raptor_uri* uri, raptor_uri* name_uri, int flags);

Expression API changes

rasqal_op enumeration gains values for LAQRS (possible SPARQL 1.1) expression operators: RASQAL_EXPR_SUM, RASQAL_EXPR_AVG, RASQAL_EXPR_MIN and RASQAL_EXPR_MAX. It also gains RASQAL_EXPR_COALESCE for LAQRS experimental coalesce expression COALESCE(expression list)

API CHANGE: The rasqal_expression structure gains a new world reference pointer field.

API CHANGE: All rasqal_expression constructors gain an initial rasqal_world argument:

  rasqal_expression* rasqal_new_0op_expression(rasqal_world* world,
    rasqal_op op);
  rasqal_expression* rasqal_new_1op_expression(rasqal_world* world,
    rasqal_op op, rasqal_expression* arg);
  rasqal_expression* rasqal_new_2op_expression(rasqal_world* world,
    rasqal_op op, rasqal_expression* arg1, rasqal_expression* arg2);
  rasqal_expression* rasqal_new_3op_expression(rasqal_world* world,
    rasqal_op op, rasqal_expression* arg1,  rasqal_expression* arg2,
    rasqal_expression* arg3);
  rasqal_expression* rasqal_new_string_op_expression(rasqal_world* world,
    rasqal_op op, rasqal_expression* arg1, rasqal_literal* literal);
  rasqal_expression* rasqal_new_literal_expression(rasqal_world* world,
    rasqal_literal* literal);
  rasqal_expression* rasqal_new_function_expression(rasqal_world* world,
    raptor_uri* name, raptor_sequence* args);
  rasqal_expression* rasqal_new_cast_expression(rasqal_world* world,
    raptor_uri* name, rasqal_expression *value);

API CHANGE: rasqal_expression_evaluate() gains an initial rasqal_world argument:

  rasqal_literal* rasqal_expression_evaluate2(rasqal_world *world,
    raptor_locator *locator, rasqal_expression* e, int flags);

Added rasqal_graph_pattern_get_origin() to get the graph from a GRAPH graph pattern:

  rasqal_literal* rasqal_graph_pattern_get_origin(rasqal_graph_pattern* gp);

Added rasqal_graph_pattern_set_filter_expression() and rasqal_graph_pattern_get_filter_expression() methods to set/get an expression for a filter graph pattern:

  int rasqal_graph_pattern_set_filter_expression(rasqal_graph_pattern* gp,
    rasqal_expression* expr);

  rasqal_expression*
    rasqal_graph_pattern_get_filter_expression(rasqal_graph_pattern* gp);

Graph Pattern API changes

rasqal_graph_pattern_operator enumeration gains RASQAL_GRAPH_PATTERN_OPERATOR_FILTER for a filter graph pattern and RASQAL_GRAPH_PATTERN_OPERATOR_LET for LAQRS experimental assignment 'graph pattern'.

API CHANGE: Removed functions rasqal_graph_pattern_add_constraint(), rasqal_graph_pattern_get_constraint_sequence() and rasqal_graph_pattern_get_constraint(). These are replaced by filter graph pattern with expression - functions rasqal_graph_pattern_set_filter_expression and rasqal_graph_pattern_get_filter_expression().

LAQRS Changes

LAQRS is now available in the configuration and does not need to be explicitly selected with configure.

Add SUM(), AVG(), MIN() and MAX() aggregate expressions to LAQRS. Supported in syntax only, not in any of the query engines.

Add LET ?var := expression experimental syntax. Query engine 2 can execute this.

Add COALESCE(expression list) experimental expression to return the first defined expression value in the list. Query engines 1 and 2 can execute this.

Literal API changes

API CHANGE: Reordered rasqal_literal_type enumeration to match numeric type promotion order: float is now between integer and double. Also added RASQAL_LITERAL_XSD_STRING for xsd:string typed literals and RASQAL_LITERAL_UDT for unknown data typed literals. Both were separated out of RASQAL_LITERAL_STRING which is now for RDF plain literals and plain/typed turned into RDF terms only

Added rasqal_literal_get_rdf_term_type() method to return the RDF term type of a literal as a rasqal_literal_type - uri, blank node, literal or cannot be represented (e.g. qname or internal literal types).

  rasqal_literal_type rasqal_literal_get_rdf_term_type(rasqal_literal* l);

Added rasqal_literal_same_term() method to perform SPARQL sameTerm() comparison.

  int rasqal_literal_same_term(rasqal_literal* l1, rasqal_literal* l2);

Added rasqal_literal_is_rdf_literal() method to check if a literal is an RDF literal (not qname or other internal types).

  int rasqal_literal_is_rdf_literal(rasqal_literal* l);

Prefix API changes

API CHANGE: The rasqal_prefix structure gain a new world reference pointer field.

API CHANGE: rasqal_new_prefix() gains an initial rasqal_world argument:

  rasqal_prefix* rasqal_new_prefix(rasqal_world* world,
    const unsigned char* prefix, raptor_uri* uri);

Query API changes

Added rasqal_query_dataset_contains_named_graph() method to check if a named graph is in a dataset:

  int rasqal_query_dataset_contains_named_graph(rasqal_query* query,
                                                raptor_uri *graph_uri);

Query Formatter Changes

SPARQL DESCRIBE no longer writes commas between terms.

Query Results API changes

Added rasqal_query_results_get_query() method to get the associated query:

  rasqal_query*
    rasqal_query_results_get_query(rasqal_query_results* query_results);

Added rasqal_graph_pattern_variable_bound_in() method to check if a graph pattern in a query defines a variable

  int rasqal_graph_pattern_variable_bound_in(rasqal_graph_pattern *gp,
    rasqal_variable *v);

Query Results Formatter Changes

SPARQL/JSON: Do not emit double " for distinct/ordered flags. Fixes Issue #0000279

Added comma-separated values (CSV), tab-separated values (TSV) and text table output formats with names csv, tsv and table.

RDQL Changes

Some updates to handle refactored query list of constraint expressions switching to a filter graph pattern. RDQL only has 0 or 1 such expressions in a single query.

SPARQL Changes

Ensure GroupOrUnionGraphPattern rule constructs the sequence of graph patterns in document order.

OptionalGraphPattern rule is now a list of 1 group graph patterns.

Fix support for <> URI literal.

Support \b, \f, \' and \" escapes in all types of quoted literals.

Fixed optional WHERE in parser to match SPARQL grammar - optional for DESCRIBE, but required for SELECT, CONSTRUCT and ASK queries.

Triples Source API changes

API CHANGE: The rasqal_triples_source and rasqal_triples_source_factory structures gain API version fields that must be set to a known API version.

API CHANGE: The rasqal_set_triples_source_factory() function for registering a triples source factory now take a function with an int error return code, and it returns an int now:

  int rasqal_set_triples_source_factory(rasqal_world* world,
     rasqal_triples_source_factory_register_fn register_fn,
     void* user_data);

Variable API changes

API CHANGE: The rasqal_variable structure gains a variables table pointer.

Other API changes

rasqal_triple_parts adds value RASQAL_TRIPLE_NONE for no parts.

rasqal_compare_flags enumeration gains RASQAL_COMPARE_URI - now also provides RDQL string to bool comparison.

API CHANGE: rasqal_new_xsd_decimal() gains an initial rasqal_world argument:

  rasqal_xsd_decimal* rasqal_new_xsd_decimal(rasqal_world* world);

Removed rasqal_triple_flags enumeration with unused values: RASQAL_TRIPLE_FLAGS_EXACT, RASQAL_TRIPLE_FLAGS_OPTIONAL and RASQAL_TRIPLE_FLAGS_LAST

Configure and Build Changes

configure now prefers pkg-config raptor to raptor-config but will accept both for now. pkg-config will not allow compiling against raptor in source tree in future.

Added --with-query-engine-version to configure to allow selecting default query engine version - values 1 or 2. This is allowed for people to switch to old engine if necessary for some existing query that the new engine cannot handle. This option will be removed at some point.

Add SHAVE support (git clone git://git.lespiau.name/shave ) to "make autotools output sane" which amounts to much less verbose messages when compiling. It is now enabled for the maintainer.

rasqal-config(1) is now a wrapper around pkg-config rasqal. Consequently --prefix=DIR and --libtool-libs are ignored.

Query Engine Changes

An entirely new query execution engine was written. The existing query execution engine was refactored out of a mess of rasqal_query and rasqal_query_results code and data. It is the default query engine as of Rasqal 0.9.17

The query engine to use can be selected at configure time wiht --with-query-engine-version to allow switching to old engine if necessary for some existing query that the new engine cannot handle. This option will be removed at some point.

The interface between the query API and the query engine is now via the rasqal_rowsource abstraction - an evaluation of a query returns a rowsource that generates the result rows of variable bindings.

Rewrote query_results class to be a pure wrapper around the rowsource and variables table, handling such things like generating triples (CONSTRUCT) from rows, creating a boolean result (ASK) by counting at least 1 result rows and generating syntax results using a query results formatter.

Added a new rasqal_query_execution_factory abstraction to provide the interface that the query execution engines implement to return a rowsource to get the results.

Both query engines can potentially be selected at runtime from the query API but ONLY when rasqal is built for development (configure --enable-maintainer-mode).

  1. If the envariable RASQAL_DEBUG_ENGINE is set with value 1 or 2, the query execution engine is chosen.
  2. The roqet(1) utility gains a -g / --engine argument.

Query engine 1 (existing):
Existing lazy evaluated engine that is hard to refactor. Internally, implemented by functions named rasqal_query_engine_1_...(). Execution factory instance rasqal_query_engine_1

Query engine 2 (new):
Based on transforming a tree of SPARQL algebra nodes constructed from the query, into a tree of rowsource-generating expressions that when evaluated at the top, returns the result rows. Each algebra node type and each rowsource type has it's own set of module unit tests built-in. Execution factory instance rasqal_query_engine_algebra

Removed query engine core legacy support for CONSTRUCT * - never in the SPARQL 1.0 language.

Pulled out row sorting code from query engine 1 into a rasqal_engine_sort module so it can be used by all query engines.

Implemented rowsource abstraction rasqal_rowsource for the following types of rowsources:

Renamed row abstraction from rasqal_query_result_row to rasqal_row since it is independent of query results and shorter.

Use new rasqal_engine_error enumeration for query engine/internal errors - OK, FAILED or FINISHED rather than altering rasqal_query_results object fields directly.

Documented query execution engine internals as autodocs.

Added a query algebra abstraction rasqal_algebra with node type rasqal_algebra_node_operator based on SPARQL algebra operators: BGP, FILTER, JOIN, DIFF, LEFTJOIN, UNION, TOLIST, ORDERBY, PROJECT, DISTINCT, REDUCED, SLICE and GRAPH. Some of these are not needed by the query execution engines since they are handled by the high level query API - such as SLICE (limit and offset) and TOLIST (turning results into user data)

Added a variables table abstraction rasqal_variables_table pulled out of the internal query results and query structures which now point to it. Constructor rasqal_new_variables_table(), destructor rasqal_free_variables_table() and methods rasqal_variables_table_add(), rasqal_variables_table_get(), rasqal_variables_table_has(), rasqal_variables_table_set(), rasqal_variables_table_get_value(), rasqal_variables_table_get_named_variables_count(), rasqal_variables_table_get_anonymous_variables_count(), rasqal_variables_table_get_total_variables_count(), rasqal_variables_table_get_named_variables_sequence() and rasqal_variables_table_get_anonymous_variables_sequence().

Allow queries with no graph pattern to be accepted such as SPARQL DESCRIBE uri (which the query engine does not execute at present but now is at least parsed).

Do not warn about unbound variables for variables bound in GRAPH and SELECT expressions.

Renamed query language-specific factory class for creating query structure from SPARQL and RDQL syntaxes from struct rasqal_query_engine_factory to rasqal_query_language_factory since it is about the language, not the engine.

Query engine known problems:

Summary of the approved DAWG SPARQL 1.0 tests as of Rasqal SVN r15841

Fail:       22 (4.99 %)
    algebra                        4
    dataset                        3
    distinct                       4
    graph                          2
    open-world                     3
    optional                       4
    reduced                        2
Pass:      419 (95.01 %)

The above results in EARL: RDF/XML and Turtle

Internal Changes

Added sparql algebra tests based on the examples in the SPARQL 1.0 Query Language Specification section 12.2.1 Converting Graph Patterns

Optionally support the experimental raptor V2 API.

Lots of resiliency fixes in the case of out of memory - Lauri Aalto.

Adjusted internal test suite to use a perl improve script that generates and uses an RDF-based manifest. The test running is now much more consise when successful.

Switch to final DAWG approved testsuite files from http://www.w3.org/2001/sw/DataAccess/tests/r2/ at CVS Revision 1.6 1008

Rasqal 0.9.16 Changes

WARNING: ABI AND API CHANGED IN THIS RELEASE.

This means: functions were removed, modified and added.

Consequently the shared library major soname version has changed from 0 to 1.

The header files moved to be installed in PREFIX/include/rasqal such as /usr/include/rasqal for usual installs.

NOTE: In the next release of rasqal, the rasqal-config utility that provides the compile and link args will be deprecated, generate a warning when run and will turn into a wrapper for pkg-config rasqal ... It will be removed in a future release.

Major Changes (break ABI and API)

Removed rasqal_init() and rasqal_finish() functions.

Added a new rasqal_world object to manage library allocations and classes with new library constructor/initializing function rasqal_world *rasqal_new_world(void) and destructor/cleanup function: void rasqal_free_world(rasqal_world*)

The following functions now take a librdf_world* world argument as the first argument: rasqal_language_name_check(), rasqal_languages_enumerate(), rasqal_new_query(), rasqal_query_results_formats_check (), rasqal_new_query_results_formatter(), rasqal_new_query_results_formatter_by_mime_type(), rasqal_new_integer_literal(), rasqal_new_typed_literal(), rasqal_new_double_literal(), rasqal_new_float_literal(), rasqal_new_uri_literal(), rasqal_new_pattern_literal(), rasqal_new_string_literal(), rasqal_new_simple_literal(), rasqal_new_boolean_literal(), rasqal_new_variable_literal(), rasqal_new_decimal_literal(), rasqal_new_decimal_literal_from_decimal()

Removed deprecated rasqal_new_floating_literal(), rasqal_graph_pattern_get_flags(), rasqal_expression_foreach() functions, RASQAL_LITERAL_FLOATING macro and rasqal_expression_foreach_fn typedef.

Modified rasqal_triples_match struct to add a rasqal_world* world field.

There is no more static data such as error pointers, arrays of query languages, URI objects; it is all attached to the rasqal_world object.

Misc changes

Fixed memory leaks in lang() expression evaluation. (Lauri Aalto)

Made several low-memory resiliance fixes in query construction and execution. (Lauri Aalto)

Library requires raptor 1.4.17 or newer to get the newly public raptor SAX2 API for reading SPARQL XML Results format.

Updated mpfr configure check to look for modern functions rather than functions that are also in the older gmp-mpfr library too.

Only the first error is reported for syntax errors in lex/parse of SPARQL and RDQL since the lack of error recovery means that the consequent messages are of little use.

Fatal errors in SPARQL and RDQL lexer/parser no longer does an abort, but exits via setjmp/longjmp.

Query Engine Changes

Added internal rowsource class to get rows from an execution engine or a syntax (query result formatter).

Graph Pattern Class Changes

Modified rasqal_graph_pattern_add_sub_graph_pattern() to have an int return value for catching allocation failures.

Query Class Changes

Modified rasqal_query_add_variable() and rasqal_query_add_prefix() to have int return values for catching allocation failures.

Query Results Class Changes

Updated the internals to allow construction of a query result outside the query engine, such as from a syntax.

Added rasqal_query_results_read() to read a query results from an iostream in some format.

Modified rasqal_query_results_formats_enumerate() to add a flags argument with RASQAL_QUERY_RESULTS_FORMAT_FLAG_READER and RASQAL_QUERY_RESULTS_FORMAT_FLAG_WRITER choices to choose formats for read or write enumeration.

Removed function rasqal_query_results_formats_enumerate_full now handled by rasqal_query_results_formats_enumerate().

Added rasqal_query_results_formatter_read() to read query results from a syntax via an iostream.

Variable Class Changes

Added rasqal_new_variable_from_variable() copy constructor.

Rasqal 0.9.15 Changes

General Changes

Many updates for better SPARQL support. Rasqal 0.9.15 passes over 92% of the DAWG SPARQL Test Suite as of 2007-11-13.

Rasqal still does not support either UNION or GROUP and OPTIONAL remains buggy. There remain a few other minor areas not yet completely supported such as XSD date and XSD dateTime equality and comparisons.

WARNING: ABI AND API CHANGE in Rasqal 0.9.16. The next release of Rasqal will be incompatible with this one.

It will have both ABI and API changes that will require source updates.

The shared library major soname will change from 0 to 1.

The header files will move to be installed in PREFIX/include/rasqal such as /usr/include/rasqal for usual installs.

The full details of the API change have not been decided but the driving change is that using the library will be done via a new Rasqal world object,k will now be used to working like a Redland librdf_world. All constructors will take this world object as an argument and all static data will be indirected from the object. Handlers such as the uri_handler/data and error handlers will also move to be set/get via the world object.

There are almost certainly other API changes and cleanups that will be done, and some possibly some of the data structures may be made internal.

Fixed issues:

SPARQL Query Language Changes

Many updates to support the SPARQL Candidate Recommendation and Proposed Recommendation versions of the language including:

Query Engine Changes

Many internal changes were made including refactoring the engine core and adding of additional internal documentation for the engine functions.

Substantial improvements were made to datatypes support:

Improved the way that literal equality and comparisons work especially with respect to datatypes. Triple pattern matching (equality) works with RDF terms. SPARQL FILTER literal equality and comparisons follow SPARQL rules with possible numeric type promotion, then using RDF Term if the types are different. ORDER comparison or DISTINCT equality checks use RDF Term. RDQL execution uses it's own existing "stringy" rules.

Set the query base URI.

Make queries that bind the origin (SPARQL GRAPH) match correctly against the default graph.

Memory reduction in stack usage when evaluation expressions (such as SPARQL FILTERs).

Public API changes

Added rasqal_home_url_string static constant.

Added rasqal_xsd_decimal class and API calls described below.

Added a decimal part to the rasqal_literal union value for RASQAL_LITERAL_DECIMAL types.

Added a parent_type part to the rasqal_literal when the type can be promoted.

Added RASQAL_EXPR_SAMETERM expression

Added RASQAL_COMPARE_RDF and RASQAL_COMPARE_URI comparison types for RDF Term comparsion rules and allowing URI comparison (without STR()) respectively.

Added rasqal_new_typed_literal() to make an RDF typed literal from a type and string (no language).

Added rasqal_new_float_literal() for wrapping a C float as XSD float.

Added new decimal literal constructor rasqal_literal* rasqal_new_decimal_literal_from_decimal(const unsigned char *string, rasqal_ xsd_decimal* decimal)

Added rasqal_literal_datatype method to get the datatype URI of a literal.

Added rasqal_literal_value method to get the literal value of a literal, looking up any variables to their value. FIXME should this not involve query results?

Added addition rasqal_triple_parts enums: RASQAL_TRIPLE_GRAPH for just the named graph part RASQAL_TRIPLE_SPO for subject, predicate and objects and RASQAL_TRIPLE_SPOG for all parts including named grpah.

Decimal Class

A new rasqal_xsd_decimal class was added to handle XSD decimals with accurate decimal arithmetic.

Constructor:

rasqal_xsd_decimal* rasqal_new_xsd_decimal(void);

Destructor:

void rasqal_free_xsd_decimal(rasqal_xsd_decimal* dec);

Methods:

double rasqal_xsd_decimal_get_double(rasqal_xsd_decimal* dec);
int rasqal_xsd_decimal_set_double(rasqal_xsd_decimal* dec, double d);
iint rasqal_xsd_decimal_set_string(rasqal_xsd_decimal* dec, const char* string);
int rasqal_xsd_decimal_set_long(rasqal_xsd_decimal* dec, long l);
int rasqal_xsd_decimal_print(rasqal_xsd_decimal* dec, FILE* stream);
char* rasqal_xsd_decimal_as_string(rasqal_xsd_decimal* dec);
char* rasqal_xsd_decimal_as_counted_string(rasqal_xsd_decimal* dec, size_t* len_p);

Numeric methods (writing to the result argument):

int rasqal_xsd_decimal_add(rasqal_xsd_decimal* result,
                           rasqal_xsd_decimal* a, rasqal_xsd_decimal* b);
int rasqal_xsd_decimal_subtract(rasqal_xsd_decimal* result,
                                rasqal_xsd_decimal* a, rasqal_xsd_decimal* b);
int rasqal_xsd_decimal_multiply(rasqal_xsd_decimal* result,
                                rasqal_xsd_decimal* a, rasqal_xsd_decimal* b);
int rasqal_xsd_decimal_divide(rasqal_xsd_decimal* result,
                              rasqal_xsd_decimal* a, rasqal_xsd_decimal* b);
int rasqal_xsd_decimal_negate(rasqal_xsd_decimal* result,
                              rasqal_xsd_decimal* a);

Comparison methods:

int rasqal_xsd_decimal_compare(rasqal_xsd_decimal* a, rasqal_xsd_decimal* b);
int rasqal_xsd_decimal_equals(rasqal_xsd_decimal* a, rasqal_xsd_decimal* b);
int rasqal_xsd_decimal_is_zero(rasqal_xsd_decimal* d);

Query Results Formatter Class Changes

Updated to the 4th version of the SPARQL XML Results format by removing the ordered and distict attributes. Removed support for all the earlier versions 1 to 3.

roqet Utility Changes

Added -D / --data options for specifying data URis for the default graph and -G / --named for URIs of named graphs (the existing -s is a also a synonym for the latter).

Other Changes

Many internal fixes and cleanups after failures so that memory is not lost and/or rasqal does not crash. These many changes were primarily made by Lauri Aalto.

Several portability fixes for helping with low-memory, embedded systems and using a compiler that is not gcc.

The SPARQL test runner check-sparql was improved to read the SPARQL XML Results format (but not via XML::Simple because it is insanely complex and counterintuitive), handle DAWG test manifests and generate EARL reports. It also has new help and usage messages.

Rasqal 0.9.14 Changes

General Changes

Added syntax and API support (but no query engine support) for experimental extensions to SPARQL called LAQRS: LAQRS Adds to Querying RDF in SPARQL.

Updated the query results formatter to allow better introspection of supported formats.

Fixed issues:

LAQRS Adds to Querying RDF in SPARQL (LAQRS)

LAQRS is experimental and Rasqal provides only syntax and API support (but no query engine support) for:

Repeating THERE IS NO QUERY ENGINE SUPPORT FOR ANY OF THE ABOVE

LAQRS is not enabled by default in Rasqal (unless using maintainer mode or building from Subversion). To enable it, configure Rasqal like this:

$ ./configure '--enable-query-languages=sparql rdql laqrs'
...

Rasqal build summary:
  RDF query languages available : rdql sparql
  RDF query languages enabled   : sparql rdql laqrs
  Triples source                : raptor 1.4.15
$

The API changes for LAQRS are as follows:

Added new query verbs RASQAL_QUERY_VERB_DELETE and RASQAL_QUERY_VERB_INSERT

Added a constructor to create a constant or 0-operand expression (for COUNT):

rasqal_expression* rasqal_new_0op_expression(rasqal_op op);

Added a query method to get the explain flag:

int rasqal_query_get_explain(rasqal_query* query);

Added a query method to get the 'group by' condition expressions:

raptor_sequence*
rasqal_query_get_group_conditions_sequence(rasqal_query* query);

rasqal_expression*
rasqal_query_get_group_condition(rasqal_query* query, int idx);

Added rasqal_expression expression types:

rasqal_variable now has a rasqal_expression* field expression.

Configuration and Build Changes

The pkgconfig rasqal.pc file and the rasqal-config script now separates directly linked shared libraries and indirectly linked ones. This prevents unnecessary references to libraries. The regular libraries includes -L plus -lrasqal and the dynamic and static libraries are available with:

So for example to statically link with rasqal, use either:

$ cc .. -o foo foo.c `pkgconfig rasqal --static --libs`

$ cc .. -o foo foo.c `rasqal-config --libs --private-libs`

Query Results Formatter Changes

Made it easier to discover and create of query results in a syntax by using a MIME Type.

Added query results formatter constructor by required mime type:

rasqal_query_results_formatter*
rasqal_new_query_results_formatter_by_mime_type(const char *mime_type);

Added a method to return the mime type of a selected formatter:

const char*
rasqal_query_results_formatter_get_mime_type(
  rasqal_query_results_formatter *formatter);

Added another query results formatter static method to enumerate possible results formats, their names, labels and (new) mime type.

int
rasqal_query_results_formats_enumerate_full(const unsigned int counter,
  const char **name, const char **label,
  const unsigned char **uri_string, const char **mime_type);

Added a static method to check for support for a query results format.

int rasqal_query_results_formats_check(const char *name,
  raptor_uri* uri, const char *mime_type);

SPARQL Query Language Changes

Updated for the SPARQL Query Language for RDF, W3C Working Draft 26 March 2007

Query Engine Changes

Added the possibility that a query's native result is a syntax (rather than natively bindings, boolean or triples) and added a method to check for that:

int rasqal_query_results_is_syntax(rasqal_query_results* query_results);

Many query engine internal changes, but these remain unsupported: OPTIONAL graph pattern (partially working), GROUP graph pattern (no support), UNION graph pattern (no support) and new SELECT REDUCED (no support).

roqet Utility Changes

roqet can use -r to generate a chosen result syntax for bindings results or triples (using raptor).

roqet now generates namespace declarations for triples when serializing output. The result is that they are used when the serializer handles them such as generating RDF/XML (Abbreviated) or Turtle.

Other Changes

rasqal_expression lost the unused rasqal_variable* field variable

Added exported variables to return the license string in rasqal_license_string and home page in rasqal_home_url_string

Rasqal 0.9.13 Changes

General Changes

Added a new rasqal_query_results_formatter class.

Many internal query engine changes, mostly to separate query and query_results internals so that eventually the run-time execution context is entirely separated from the static query structure.

Added query features system with one initial feature RASQAL_FEATURE_NO_NET.

Configuration and Build Changes

Allow shell variable LEX to be set to things that aren't exactly flex Recommend flex 2.5.33 and update configure to point at the main flex site whichq finally gets 2.5.33 after 9 years.

PKGCONFIG vars not substituted correctly in configure.ac. Remove @LDFLAGS from rasqal.pc.in. Fixes Issue#0000098.

Win32 build fixes.Correct win32 static linking of rasqal. (Fixes Issue#0000100) and include float.h and #define isnan on win32. (Fixes Issue#0000111).

In maintainer mode, add all the supported compiler -W warning flags to the CFLAGS.

Declare several rasqal functions with GCC printf-formatting attributes when using a new enough GCC.

Remove broken debug messages. #0000131.

SPARQL Query Language Changes

Added labels to grammar tokens using bison to give better error messages.

Fix looking for EOF in an unsigned variable since it may be negative on some systems.

The query engine now implements SPARQL GRAPH matching.

RDQL Query Language Changes

Added labels to grammar tokens using bison to give better error messages.

Fix looking for EOF in an unsigned variable since it may be negative on some systems.

Query Engine Changes

The query engine now implements SPARQL GRAPH matching.

Merge only all adjacent sequences of basic graph pattern triples, not anything with triples such as optional BGP triples. Fixes Issue#0000080.

When an exact triple match is present, do not execute a query forever. Fixes Issue#